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This  report  presents  the  1978  evaluation  of  the  Professional 
Standards  Review  Organization  (PSRO)  program  conducted  by  the  Health 
Care  Financing  Administration  (HCFA).  The  study  shows  PSROs  have 
reduced  Medicare  hospital  utilization,  relative  to  non-PSRO  areas, 
to  the  point  where  the  program's  concurrent  review  activity  now  pays 
for  itself. 

Eighty-one  of  the  96  PSROs  studied  saved  Medicare  days  of 
hospital  care  relative  to  non-PSRO  areas.  In  29  PSROs,  the  value 
of  the  hospital  days  saved  exceeded  the  costs  of  carrying  out 
concurrent  review  and,  in  five  PSROs,  these  savings  were  more  than 
double  the  costs.  In  addition,  PSROs  are  making  an  impact  in  low 
hospital-use  areas,  such  as  California,  as  well  as  in  high  hospital- 
use  areas  such  as  New  York  City. 

The  PSRO  program  is  becoming  an  effective  partner  in  the  HEW 
campaign  to  reduce  unnecessary  hospital  care  and  to  assure  quality 
of  care.  In  1977,  $45  million  was  spent  on  concurrent  review  for 
Medicare  patients  by  the  96  PSROs  studied,  resulting  in  savings 
of  about  $50  million  from  the  elimination  of  unnecessary  hospital 
days. 

PSROs  have  now  demonstrated  that  they  can  reduce  the  number 
of  hospital  days  for  Medicare  beneficiaries  relative  to  non-PSRO 
areas.  Yet  the  savings  to  the  Medicare  program  from  each  day  saved 
is  approximately  40  percent  of  the  cost  paid  by  the  program  for 
an  actual  day  of  care.  This  is  because  an  empty  bed  is  still 
expensive;  fixed  costs  associated  with  empty  beds  comprise  60 
percent  of  the  per  diem  costs.  When  unnecessary  beds  are  closed  or 
are  converted  to  much-needed  long  term  care  beds,  then  Medicare  and 
Medicaid  will  be  able  to  reap  much  greater  savings  from  the  accom- 
plishments of  PSROs. 

Equally  important  is  the  fact  that  some  PSROs  are  clearly 
doing  a  much  better  job  than  others,  and  that  a  PSRO's  impact 
in  reducing  unnecessary  hospitalization  seems  to  increase  with 
maturity.  We  are  striving  to  bring  the  entire  country  up  to  the 
accomplishments  and  level  of  savings  demonstrated  by  PSROs  in  the 
states  of  New  York,  California  and  Connecticut.  Furthermore,  this 
evaluation  and  future  ones  like  it  will  enable  us  to  discontinue 
support  of  ineffective  PSROs,  boosting  the  impact  of  review  still 
further. 

We  expect  to  continue  the  initiatives  begun  this  past  year 
to  improve  the  program  performance  of  all  PSROs,  particularly  those 
which  have  the  furthest  to  come.  The  emphasis  in  our  management 
improvement  program  will  continue  in  two  key  areas : 
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—  Shifting  from  a  process  to  outcome  orientation  via  the 
establishment  of  performance  objectives  by  PSROs. 

—  Improving  program  efficiency  by  reducing  the  costs  of 
review » 

PSROs  with  implementation  in  at  least  90  percent  of 
their  hospitals  most  often  had  unit  costs  of  review 
of  $9-$10  in  1977.  This  compares  favorably  with  the 
target  rate  we  have  set  for  the  program  funding  of  an 
average  of  $8.70  per  discharge  by  the  end  of  Fiscal  Year 
1979. 

Perhaps  the  most  encouraging  aspect  of  the  1978  PSRO  evaluation 
is  its  documentation  of  the  considerable  progress  in  the  program 
between  1976  and  1977,  which  occurred  before  most  of  these  hew 
management  initiatives  were  begun.  The  fact  that  a  keystone  of  the 
program  —  concurrent  review  —  was  able  to  pay  for  itself  so  early 
in    its    development    is    a   particularly    noteworthy  accomplishment. 

HCFA  appreciates  the  cooperation  shown  by  the  many  physicians 
and  review  coordinators  throughout  the  country  who  have  been  work- 
ing through  the  PSRO  program  to  contain  health  care  costs.  Cer- 
tainly, without  their  commitment  to  this  program,  the  study's 
findings  would  not  have  been  so  positive.  The  progress  demonstrated 
in  the  PSRO  program  shows  that  responsible,  independent  physicians 
can  form  an  effective  partnership  with  the  Federal  Government.  We 
must  now  complete  the  job  —  to  reduce  still  further  the  escalation 
in  Federal  health  care  costs  and  to  intensify  our  efforts  to  improve 
quality  of  care. 


Leonard  D.  Schaeffer 
Administrator 

Health  Care  Financing  Administration 
Department  of  Health,  Education 
and  Welfare 
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ABSTRACT 


This  volume  presents  the  1978  evaluation  of  the  Professional 
Standards  Review  Organization  (PSRO)  program,  prepared  by  the  Office 
of  Policy,  Planning  and  Research  of  the  Health  Care  Financing 
Administration.  It  is  the  second  report  of  a  continuing  evaluation 
of  the  PSRO  program.  The  evaluation  focused  on  utilization  control, 
program  costs,  quality  assurance,  program  management  and  program 
development.  A  benefit-cost  analysis  of  concurrent  review  was  also 
performed.  The  evaluation's  analvses  indicate  what  impact  the  PSRO 
program  had  achieved  by  the  end  of  1977,  provide  continued  develop- 
ment of  baseline  data  for  future  evaluations  and  provide  pilot  tests 
of  numerous  evaluation  methodologies  related  to  utilization  control 
and  quality  assurance.  New  to  the  1978  evaluation  are  studies 
related  to  diagnostic  and  surgical  procedure  data,  the  PSRO  Hospital 
Discharge  Data  Set  (PHDDS) ,  Medical  Care  Evaluation  Study  impact, 
and  program  management.  Since  the  evaluation  is  based  on  calendar 
year  1977  data,  it  does  not  reflect  1978  program  impact. 

The  Medicare  utilization  study,  the  major  effort  of  the  1978 
evaluation,  investigated  the  impact  of  the  PSRO  program  on  days  of 
hospital  care  per  1,000  Medicare  enrollees,  hospital  discharges  per 
1,000  enrollees  and  average  length  of  stay.  PSRO  program  impact  was 
estimated  by  comparing  hospital  utilization  between  96  active  and  93 
non-active  PSRO  areas  for  1977.  The  analysis  indicated  a  statistic- 
ally significant  reduction  for  days  of  care/1,000  and  discharges/ 
1,000  enrollees,  but  not  for  average  length  of  stay.  The  reduction 
of  1.5  percent  in  days  of  care/ 1,000  enrollees  attributable  to  PSRO 
impact  is  one  of  the  important  findings  of  the  study  because  of  its 
direct  implication  for  lower  Medicare  program  expenditures.  All  but 
fifteen  of  the  96  active  PSROs  showed  a  decline  in  hospital  days  per 
1,000  enrollees  relative  to  inactive  PSROs.  Individual  PSRO  effects 
range  from  an  8.75  percent  decline  to  a  1.95  percent  increase  in 
days  of  care.  Analysis  of  PSRO  impact  by  U.S.  Census  region  showed 
that  PSROs  in  the  Northeast  and  West  had  aggregate  reductions  in 
days  of  care  per  1  ,000  enrollees  of  2.7  percent  and  2.6  percent, 
respectively,  whereas  PSROs  in  the  North  Central  and  South  had  an 
insignificant  impact  on  utilization  rates  in  their  regions. 

Three  hospital  utilization  studies  of  a  more  developmental 
nature  were  also  conducted.  One  study  compared  diagnostic  and 
surgical  procedure  data  drawn  from  the  20  percent  Medicare  discharge 
file  for  the  years  1971-1973  and  1976.  This  study  hypothesized  that 
the  PSRO  program  might  have  a  differential  impact  on  selected 
diagnoses  and  procedures.  The  study  generally  confirmed  the  finding 
of  the  1977  PSRO  evaluation  that  PSROs  had  exerted  little  influence 
on  hospital  utilization  by  the  end  of  1976.  Continuing  this  study 
with  1977  data  should  permit  a  more  thorough  examination  of  program 
impact  on  selected  diagnoses. 
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The  other  two  hospital  utilization  studies  were  based  on  data 
available  for  1976  and  1977  from  PHDDS  which  includes  discharge 
information  collected  by  the  PSROs.  One  PHDDS  study  compared 
average  lengths  of  stay  obtained  from  the  Medicare  20  percent  sample 
discharge  file  with  comparable  estimates  derived  from  the  PHDDS. 
Aggregate  length  of  stay  estimates  were  found  to  be  similar,  but 
diagnostic  and  procedural  average  length  of  stay  estimates  were 
highly  dissimilar.  Comparison  of  aggregate  lengths  of  stay  for  17 
PSROs  from  1974  to  1977  found  no  significant  changes  over  time  in 
this  statistic.  The  other  PHDDS  study  examined  changes  in  the  level 
of  case  severity  for  eight  active  PSROs.  The  findings  suggest  that 
case  severity  may  be  increasing  in  PSRO  hospitals.  However,  it  was 
not  possible  to  draw  firm  conclusions  because  no  control  data  were 
available  from  inactive  PSROs;  the  alternative  hypothesis  that  these 
changes  are  related  to  improved  reporting  procedures  has  not  yet 
been  adequately  addressed. 

The  study  of  PSRO  program  costs  examined  variations  in  unit 
costs,  correlations  between  delegated  hospital  costs  and  hospital 
characteristics  and  the  costs  of  a  fully  implemented  PSRO  program. 
No  attempt  was  made  to  estimate  the  incremental  costs  of  FSRO 
review  over  and  above  those  of  hospital-based  utilization  review,  or 
the  program  costs  to  be  expected  if  focused  review  were  implemented. 
All  cost  data  were  esimated  on  a  calendar  year  1977  basis  so  that 
they  could  be  compared  with  calendar  year  1977  utilization  impact 
estimates. 

The  national  average  cost  for  total  hospital  review  in  hos- 
pitals which  were  performing  PSRO  review  at  the  end  of  fiscal  year 
1977  was  estimated  at  $12.08  per  discharge  subject  to  review.  The 
range  was  $6.52  to  $74.90.  Among  the  42  PSROs  that  /had  review 
implemented  in  more  than  90%  of  their  hospitals,  unit  costs  for 
total  hospital  review  showed  a  mode  between  $9  and  $10  with  approxi- 
mately 50%  falling  in  the  range  between  $8  and  $12.  Total  hospital 
review  includes  concurrent  review,  medical  care  evaluation  studies 
and  areawide  review.  Unit  concurrent  review  costs  varied  from 
$3.25  to  $18.09.  The  median  unit  cost  for  delegated  concurrent 
review  was  $8.76;  for  non-delegated  concurrent  review  the  median 
unit  cost  was  $10.94.  Total  expenditures  needed  to  fund  a  fully 
implemented  PSRO  program  during  fiscal  year  1977  were  estimated  at 
approximately  $235  million. 

The  benefit-cost  analysis  of  concurrent  review  estimated 
a  dollar  value  for  utilization  changes  attributable  to  concurrent 
review  and  compared  this  value  to  the  estimated  costs  of  concurrent 
review  to  arrive  at  an  aggregate  benefit-cost  ratio  slightly  in 
excess  of  1:1.  About  30  of  the  96  active  PSROs  studied  had  aggre- 
gated benefit-cost  ratios  in  excess  of  1:1.  Thus,  even  though  this 
ratio  reflects  a  comparison  of  full  program  costs  with  the  incre- 
mental benefits  of  PSRO  concurrent  review  (over  hospital-based 
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utilization  review),  it  shows  that  the  savings  in  Medicare  outlays 
from  lower  utilization  rates  more  than  cover  the  costs  of  operating 
the  program,  even  when  a  conservative  (incremental)  benefit  estimate 
is  used.  Further,  the  studies  show  that  while  the  PSRO  program 
cannot,  by  itself,  be  held  responsible  for  achieving  overall  hos- 
pital cost  containment  objectives,  the  program  can  be  a  meaningful 
component  linked  with  other  efforts  directed  toward  an  effective 
national  cost  containment  strategy. 

Medical  care  evaluation  studies  (MCEs)  are  the  PSRO  program's 
principal  means  of  monitoring  and  improving  the  quality  of  care  in 
hospitals  which  treat  Federal  beneficiaries.  MCEs  involve  both  an 
initial  audit,  which  compares  a  sample  of  patient  records  with 
explicit  criteria  of  good  care,  and  a  reaudit  of  care,  generally 
within    a    year,    to   ascertain    the   effects    of    recommended  changes. 

Two  approaches  were  used  to  evaluate  the  MCE  component  of  the 
PSRO  program.  The  first  developed  descriptive  statistics  about  the 
extent  to  which  MCEs  have  been  implemented  nationwide.  It  found 
that  the  number  of  initial  MCE  audits  has  generally  increased  each 
quarter  since  the  second  quarter  of  1975,  reaching  a  cumulative 
total  of  over  22,000  by  June  1978.  This  trend  represents  both  an 
increasing  number  of  PSROs  reporting  and  an  increasing  average 
number  of  MCEs  reported  per  PSRO.  Ten  PSROs  (nine  percent  of  those 
reporting)  accounted  for  35  percent  of  all  audits,  were  evenly 

distributed  beween  medical  and  surgical  topics;  the  twenty  most 
frequently  audited  topic  categories  accounted  for  43.6  percent  of 
all  studies  done.  Approximately  1,200  reaudits  were  reported 
through  June  1978,  a  much  smaller  number  than  should  have  occurred, 
given  that  70  percent  of  initial  audits  reported  findings  which 
required  reaudits.  This  suggests  that  PSRO  hospitals  are  not  yet 
receiving  sufficient  feedback  from  the  MCE  process. 

A  second  study  involving  MCEs  developed  and  tested  a  method- 
ology to  measure  the  impact  of  MCEs  on  the  medical  care  process. 
This  pilot  study  tentatively  suggests  that  there  is  a  reduction  in 
the  extent  to  which  post-MCE  medical  care  practices  depart  from 
established  criteria  of  care.  However,  the  data  generated  by  the 
MCE  process  are  not  yet  adequate  to  relate  the  MCE  process  to 
changes  in  the  quality  of  care  delivered. 

The  1978  evaluation  included  an  appraisal  and  synthesis  of  the 
results  of  26  routine  site  assessments  of  PSRO  management  perfor- 
mance conducted  by  the  Health  Standards  and  Quality  Bureau  of  the 
Health  Care  Financing  Administration.  All  26  PSROs  received 
generally  high  ratings  for  competence  of  executive  leadership,  data 
system  capability,  implementation  of  hospital  review  as  scheduled, 
and  communication  with  hospitals  regarding  hospital  performance.  On 
the  other  hand,  PSROs  tended  to  be  rated  lower  regarding  the  extent 
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to  which  they  have  withdrawn  delegation  from  hospitals,  imposed 
denials  and  sanctions,  conducted  MCE  studies,  and  carried  out 
ancillary  service  review  and  review  of  physician  services*  In 
general,  the  data  indicate  that  individual  PSRO  performance  tends  to 
improve  over  time. 

All  of  these  evaluation  studies  should  be  interpreted  with  some 
caution*  First,  the  most  current  hospital  utilization  data  are  for 
1977  and  do  not  reflect  the  considerable  changes  in  program  opera- 
tions since  1977.  The  Health  Standards  and  Quality  Bureau  (HSQB) 
has  established  program  initiatives  covering  three  areas:  (1) 
implementation  objectives,  (2)  general  management  objectives  and  (3) 
improvements  in  the  review  process*  Organizational  changes  geared 
towards  further  implementation  of  review  and  significant  efforts  to 
improve  program  performance  have  occurred*  Also,  PSRO  demonstra- 
tion projects  are  being  conducted  to  help  shape  Departmental  policy 
and  develop  review  methodologies  for  long  term  care  review  (15 
projects),  ambulatory  care  review  (5  projects),  ancillary  services 
review  (10  projects),  focused  review  (12  projects)  and  surgical 
evaluation  review  (5  projects). 

Second,  the  possibility  cannot  be  ruled  out  that  the  early 
PSROs,  for  which  data  were  available,  represent  a  self -selected 
group  of  areas  with  the  greatest  potential  for  achieving  reductions 
in  utilization  rates.  If  this  is  true,  the  results  of  the  Medicare 
utilization  study  are  biased,  that  is,  the  study  may  overestimate 
the  program's  ability  to  influence  utilization*  If  self-selection 
is,  indeed,  a  factor,  it  is  not  clear  whether  the  bias  is  strong  or 
weak;  as  the  majority  of  new  PSROs  gain  experience,  they  may  even- 
tually tend  to  become  as  effective  as  older  PSROs* 

Third,  the  impact  of  PSRO  utilization  control  activities  is 
based  solely  upon  the  hospital  utilization  experience  of  aged 
Medicare  enrollees  and  does  not  include  Medicaid  patients  or 
Medicare  disabled  beneficiaries.  However,  aged  'edicare  enrollees 
account  for  70  percent  of  all  Federal  discharges  and  an  even  greater 
proportion  of  days  of  care.  Similarly,  even  though  benefit-cost 
ratios  for  the  program  are  estimated  only  for  concurrent  review  and 
not  all  PSRO  functions,  concurrent  review  costs  constitute  approxi- 
mately 70  percent  of  program  expenditures. 

Fourth,  the  benefit-cost  analvsis  compares  total  PSRO  concur- 
rent review  cost  with  the  difference  in  hospital  utilization 
in  active  PSRO  areas  relative  to  that  in  non-PSRO  control  areas*  To 
the  extent  that  non-PSRO  utilization  review  activities  have  reduced 
hospital  utilization  in  these  comparison  areas,  the  benefit-cost 
ratio  calculated  in  the  1978  Evaluation  is  conservative*  Finally, 
the  MCE  impact  study  is  based  on  limited  samples  and  data;  generali- 
zation of  the  findings  must  await  more  conclusive  studies  now 
underway. 

The  key  findings  of  the  1978  evaluation  effort  are  summarized 

below: 
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Impact  on  Controlling  Hospital  Utilization  During  1977 

•  Ninety-six  active  PSRO  areas,  when  compared  to  93  in- 
active PSRO  areas,  were  associated  with  a  relative 
reduction  of  1.5  percent  in  hospital  days  of  care  per 
1,000  aged  Medicare  enrollees  from  1974  to  1977. 

•  Eighty-one  of  the  96  active  PSROs  examined  showed  a 
relative  decline  in  hospital  days  of  care  per  1,000  aged 
Medicare  enrollees. 

•  The  observed  relative  reduction  in  days  of  care  by  U.S. 
Census  region  were: 


Northeast 
West 

North  Central 
South 


-2.7  percent 
-2.6  percent 
-  .5  percent 
No  change 


•  The  mature  PSROs  seemed  to  be  somewhat  more  effective 
than  the  newer  PSROs. 

•  PSRO  impact  on  days  of  care  classified  by  diagnosis  and 
surgical  procedure  has  vet  to  be  demonstrated. 

•  There  is  some  evidence  that  PSRO  hospitals  are  experi- 
encing increases  in  case  severity,  that  is,  more  severe 
illnesses  are  being  treated  in  hospitals.  This  may 
indicate  a  more  efficient  use  of  hospital  services  for 
Medicare  enrollees. 

1977  Program  Costs  and  Benefit-Cost  Analysis 

•  Cost  analyses  indicated: 

—  The  average  hospital  review  unit  cost  was  $12.08  with 
a  range  of  $6.52  to  $74.90. 

For  PSROs  with  implementation  in  90  percent  of  their 
hospitals,  the  average  hospital  review  unit  cost  was 
most  often  in  the  range  $9  to  $10  with  about  5  0% 
falling  between  $8  and  $12. 

—  Median    delegated    concurrent    review    cost   was  $8.76 
per  discharge  subject  to  review. 

Median  non-delegated  concurrent  review  cost  was 
$10.94  per  discharge  subject  to  review. 

—  A  fully  implemented  PSRO  program  in  1977  would  have 
cost  approximately  $235  million. 
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•  The  benefit-cost  ratio  of  concurrent  review  was  slightly 
in  excess  of  1:1. 

•  Approximately  30  of  the  96  active  PSROs  studied  exhibited 
benefit-cost  ratios  in  excess  of  1:1. 

Quality  Assurance  Activity 

•  The  number  of  medical  care  evaluation  studies  (MCEs)  has 
been  increasing  steadily  since  the  initiation  of  the  PSRO 
program. 

•  MCE  reaudits  are  not  being  completed  at  the  frequency 
required  by  the  findings  of  the  initial  audits. 

•  MCEs  appear  to  be  improving  the  quality  of  care,  but  data 
generated  by  the  MCE  process  are  still  inadequate  for 
definitive  evaluation  purposes. 

This  19  78  PSRO  Evaluation  Report  is  organized  as  follows:  Section 
1  provides  an  introduction  and  summary  which  defines  the  1978 
evaluation  strategy  and  overall  programmatic  background,  outlines 
the  evaluation  issues  involved,  briefly  describes  the  studies 
conducted,  and  summarizes  their  results.  Sections  2  through  7  pro- 
vide detailed  descriptions  and  findings  of  the  individual  studies; 
the  topics  studied  include  utilization  impact,  program  costs, 
benefit-cost  analysis  of  concurrent  review,  MCE  studies,  program 
management,  and  overall  program  status  and  directions  respectively. 
Finally,  Section  8  discusses  preliminary  plans  for  the  1979  PSRO 
evaluation.  Several  appendices  are  included  which  detail  the 
implementation  status  of  individual  PSROs  and  provide  a  glossary  of 
terms  and  abbreviations. 
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1.      INTRODUCTION  AND  SUMMARY 


1.1  Background 

1.1.1  Purpose  of  This  Report;  This  document  presents  the 
1978  evaluation  of  the  Professional  Standards  Review  Organization 
(PSRO)  program,  prepared  by  the  Office  of  Policy,  Planning  and 
Research  (OPPR)  of  the  Health  Care  Financing  Administration,  (HCFA). 
It  is  the  second  report  of  a  continuing  Departmental  evaluation  of 
the  PSRO  program,  presenting  both  new  studies  and  updates  of  studies 
reported  in  the  1977  PSRO  evaluation.  The  1978  report  provides  more 
detail  about  study  methodologies  and  findings  than  did  the  1977  PSRO 
Evaluation  Report/ 1  in  the  expectation  that  this  will  be  more 
useful  for  those  interested  in  and  responsible  for  the  PSRO  program 
development. 

The  report  is  divided  into  three  major  parts.  The  first,  the 
introduction  and  summary,  provides  a  comprehensive  overview  of  the 
19  78  evaluation  and  a  discussion  of  the  relationships  among  indi- 
vidual studies.  The  second  part,  comprising  Sections  2  through  7, 
provides  detailed  descriptions  and  results  of  individual  studies. 
The  last  component,  Section  8,  discusses  preliminary  directions  for 
the  1979  PSRO  evaluation. 

1.1.2  PSRO  Program  Mandate 

The  PSRO  program  is  mandated  by  the  Social  Security  Amendments 
of   1972   (P.L.   92-603)   with  the  following  Declaration  of  Purpose: 

Section  1151.  In  order  to  promote  the  effective,  effi- 
cient, and  economical  delivery  of  health  care  services  of 
proper  quality  for  which  payment  may  be  made  (in  whole  or 
in  part)  under  this  Act  and  in  recognition  of  the  in- 
terests of  patients,  the  public,  practitioners  and 
providers  in  improved  health  care  services,  it  is  the 
purpose  of  this  part  to  assure,  through  the  application 
of  suitable  procedures  of  professional  standards  review, 
that  the  services  for  which  payment  may  be  made  under  the 
Social  Security  Act  will  conform  to  appropriate  profes- 
sional standards  for  the  provision  of  health  care  and 
that  payment  for  such  services  will  be  made  — 

(1)  only  when,  and  to  the  extent,  medically  necessary, 
as  determined  in  the  exercise  of  reasonable  limits 
of  professional  discretion;  and 


/1  USDHEW,  Health  Services  Administration,  Office  of  Planning, 
Evaluation  and  Legislation,  Executive  Summary,  Vol.  I  of  PSRO; 
An  Initial  Evaluation  of  the  Professional  Standards  Review 
Organization,  February  1978. 
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(2)  in  the  case  of  services  provided  by  a  hospital  or 
other  health  care  facility  on  an  inpatient  basis, 
only  when  and  for  such  period  as  such  services 
cannot,  consistent  with  professionally  recognized 
health  care  standards,  effectively  be  provided  on  an 
outpatient  basis  or  more  economically  in  an  inpa- 
tient health  care  facility  of  a  different  type,  as 
determined  in  the  exercise  of  reasonable  limits  of 
professional  discretion. /1 

This  language  provides  the  PSRO  program  with  a  dual  mandate: 
to  minimize  the  costs  of  providing  care  to  federal  beneficiaries  and 
to  assure  that  professional  standards  are  observed  in  the  provision 
of  care.  While  cost  and  quality  responsibilities  are  inextricably 
linked,  the  relative  importance  of  each  one  of  these  complementary, 
and  at  times  conflicting,  objectives  has  shifted.  The  current 
political  climate  appears  to  stress  the  utilization  control  mission 
of  the  PSRO  program  in  the  context  of  developing  a  national  strategy 
to  reduce  the  growth  of  health  care  expenditures.  This  evaluation 
does  not  attempt  to  resolve  the  cost  versus  quality  issue,  but  will 
examine  the  program's  accomplishments  in  both  domains. 

1.1.3    Evaluation  Mandate 

The  original  mandate  for  the  evaluation  of  the  PSRO  program 
called  for  the  National  Professional  Standards  Review  Council 
(NPSRO/2  to  report  to  the  Congress  and  the  Secretary,  not  less 
than  annually,  findings  relative  to  the  PSRO  program's  effective- 
ness, accomplishments  and  relative  regional  performance ./3  The 
Medicare,  Medicaid  Anti-Fraud  Amendments  of  1977  (P.  L.  95-142)/4 
deleted  the  NPSRC  requirement  to  submit  annual  reports  and  in  its 
place  required  that  the  Secretary  of  DHEW  submit  to  Congress  annual- 
ly a  full  and  complete  report  on  the  PSRO  program's  status,  admini- 
stration, impact  and  cost,  with  recommendations  for  legislative 
changes.  The  statute  specifically  calls  for  analysis  of  patterns  of 
utilization  rates  and  changes  in  medical  practices  attributable  to 
activities  of  PSRO.  The  NPSRC,  however,  is  still  required  to 
perform  studies  and  investigations  and  report  to  the  Congress.  This 
report  attempts  to  meet,  in  part,  the  needs  of  both  the  Secretary 
and  the  NPSRC. 


/I       U.S.   Congress,   The  Social  Security  Act,   Title  XI,  Section 
1151.  ~ "  " ' 

/2      A  national  council  of  physicians  mandated  by  the  original 
PSRO  statute  to  guide  the  development  of  the  PSRO  program. 

/3      U.S.   Congress,   The  Social  Security  Act,   Title  XI,  Section 
1163(f). 

/4      U.S.  Congress,  P.L.  95-142,  Section  5(k).     This  Act  is  commonly 
referred  to  as  HR3. 
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1.1.4     1977  PSRO  Operational  Profile 


The  1978  PSRO  evaluation  estimates  program  impacts  for  calendar 
year  1977.  This  subsection  presents  a  brief  summary  of  the  pro- 
gram's structure  and  selected  operational  statistics  for  that  year. 
In  order  to  meet  the  legislative  objectives  set  for  the  program, 
most  PSROs  adapted  the  model  acute  care  hospital  review  plan  de- 
veloped by  DHEW.  This  model  plan  consisted  of  three  review  compo- 
nents: (1)  concurrent  review  (CR),  (2)  medical  care  evaluation 
(MCE)  studies  and  (3)  profile  analysis.  In  concurrent  review, 
admissions  are  reviewed  against  physician-established  criteria  for 
medical  necessity.  Certified  admissions  are  assigned  an  initial 
number  of  days  according  to  local  diagnosis-specific  norms  of  care. 
Periods  of  hospitalization  extending  beyond  the  initially  certified 
number  of  days  are  reviewed  to  determine  whether  a  continued  need 
for  hospitalization  exists. /M  MCE  studies  consist  of  in-depth 
retrospective  reviews  to  determine  whether  certain  criteria  were  met 
which  are  thought  to  assure  professionally  accepted  standards  of 
care.  Profile  analysis  is  a  statistical  analysis  of  aggregate 
patient  care  data  conducted  after  patient  discharge  which  provides 
information  on  patterns  of  health  services  utilization  and  patterns 
of  care  rendered  to  patients. 

Individual  PSRO  projects  pass  through  three  stages  of  develop- 
ment and  funding.  The  planning  stage  comprises  the  development  of 
review  plans  and  organizational  structure  and  provides  for  the 
enrollment  of  physician  members.  The  conditional  stage  lasts  at 
least  24  months  and  ensures  that  the  PSRO  is  operationally  viable 
before  being  fully  designated.  Finally,  in  order  to  be  fully 
designated,  a  PSRO  must  be  satisfactorily  performing  both  acute  care 
hospital  and  long  term  care  institutional  review. 

The  PSRO  program  was  only  partially  developed  by  mid-1977  and 
its  activities  and  potential  impact  were  limited  to  acute  care 
hospitals  with  the  exception  of  certain  limited  demonstrations  in 
long  term  and  ambulatory  care  review.  Table  1  summarizes  the  PSRO 
program  implementation  status  as  of  June  30,  1977.  While  no  PSRO 
areas  were  fully  designated,  107  of  203  possible  PSROs  were  condi- 
tional, and  only  31  PSRO  areas  were  totally  unfunded.  In  the 
conditional  PSROs,  85.5  percent  of  the  hospitals  were  under  binding 
PSRO  review  covering  40.0  percent  of  the  Federally  financed  dis- 
charges (Medicare,  Medicaid,  and  Maternal  and  Child  Health  Program 
patients).  Most  hospitals  were  either  fully  or  partially  delegated 
as  opposed  to  being  non-delegated.  On  the  other  hand,  MCE  activity 
was  widespread;  as  of  June  30,  1977,  93  conditional  PSROs  had 
performed  a  total  of  9,370  studies.     Profile  analysis  was  the  least 


/ 1  Concurrent  review  (CR)  can  either  be  performed  by  the  PSRO 
(non-delegated)  or  delegated  to  the  hospital.  The  initial 
development  of  the  program  stressed  delegated  CR. 


3 


TABLE  1 


1977  PSRO  IMPLEMENTATION  STATUS 

A.  Financial  Data  (fiscal  year  1977) 

1.  Total  Program  Level/ 1;  $102,125,000 

2.  Hospital  Review  Costs:  41,000,000 

3.  Total  Appropriations  :  61,125,000 

B.  Organizational  Status  (June  30,  1977) 

Unfunded  Areas  31 
Planning  PSROs  65 
Conditional  PSROs  107 

203/2 

C.  Summary  of  Review  Activity  (June  30,  1977) 

1.  Percent   of  Hospitals  in  Conditional  Areas  Implementing 
Review  85.5 

2.  Percent    of    Federal   Discharges    in   Conditional   Areas  Under 
Review  40.0 

3.  Percent    of    Implemented   Hospitals    Which   Were    Fully  Dele- 
gated for  Concurrent  Review  68.4 

4.  Percent   of    Implemented  Hospitals   Which  Were  Partially 
Delegated  for  Concurrent  Review  7.6 

5.  Percent   of    Implemented  Hospitals   Which  Were  Non-delegated 
for  Concurrent  Review  24.0 

MCE  Activity 

Ninety-three  PSROs  had  performed  9,370  MCE  studies.  (June  30, 
1977) 


/1  Prior  to  FY  1977,  hospital  review  costs  were  paid  through 
direct  appropriations.  Hospital  review  costs  are  now  paid  from 
the  Medicare  Trust  Funds.  Total  program  funding  is  composed  of 
total  appropriations  and  Trust  Fund  payments. 

/2  Of  the  total  of  203  originally  designated  PSRO  areas,  189 
were  included  in  the  study.  Fourteen  areas  were  eliminated 
from  the  analysis  for  the  following  reasons:  eight  Los 
Angeles  PSRO  areas  were  aggregated  into  one  area  in  order  to 
compute  utilization  rates;  five  areas  were  aggregated  into  one 
in  Massachusetts;  and  three  areas  -  Kansas,  Virgin  Islands  and 
Puerto  Rico  -  were  eliminated  because  of  data  quality  pro- 
blems. Also,  Texas  was  included  in  the  analysis  as  nine  PSRO 
areas.  In  1978,  however,  Texas  had  been  redesignated  as  a 
single  statewide  PSRO.  Thus,  the  current  number  of  designated 
PSROs  is  195. 
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developed  of  the  program  components.  Insufficient  data  were 
available  to  support  an  analysis  of  the  program's  profiling  activ- 
ities. 

As  noted  in  the  1977  PSRO  Evaluation  Report,  the  program's 
impact  is  undoubtedly  influenced  by  the  activities  of  other  organi- 
zations performing  cost  containment  activities.  As  of  1977,  formal 
linkages  with  cost  commissions,  health  systems  agencies,  etc.,  were 
not  well-developed.  Continued  improvements  in  these  areas  are 
expected  to  augment  future  PSRO  utilization  control  and  quality 
enhancement  activities. 

1.2      The  1978  PSRO  Evaluation  Strategy 

The  1978  PSRO  evaluation  strategy  is  an  extension  of  the  1977 
Evaluation  in  that  its  goals  and  objectives  were  essentially  drawn 
from  the  document  Program  Evaluation  Plan:  Professional  Standards 
Review  Organization/ 1  under  the  general  guidance  of  a  governmental 
advisory  group./ 2    Areas  of  emphasis  include: 

•  Utilization  Impact 

•  Program  Costs 

•  Benefit-Cost  Analysis  of  Concurrent  Review 

•  Medical  Care  Evaluation  (MCE)  Studies 

•  Program  Management  Assessments 

•  1978  Program  Status  and  Directions 

•  Directions  for  Future  Evaluation 

Table  2  describes  the  1978  evaluation  strategy  by  listing 
component  studies  and  key  analytic  variables.  The  following  sub- 
section highlights  these  component  studies  and  their  interrelation- 
ships. 

1.2.1    Utilization  Impact 

Four  utilization  studies  were  conducted  using  the  Medicare 
claims  files  and  the  PSRO  Hospital  Discharge  Data  Set  (PHDDS).  The 
major  study  in  this  area,  the  100  percent  Medicare  claims  file 
analysis,   is  the  most  substantive  study  undertaken  in  the  1978 


/1  U.S.  DHEW,  Office  of  the  Assistant  Secretary  for  Health,  Office 
of  Professional  Standards  Review,  Program  Evaluation  Plan: 
Professional  Standards  Review  Organizations,  September  22, 
1975. 

/2  The  governmental  advisory  group  consisted  of  representatives 
from  the  Office  of  the  Assistant  Secretary  for  Planning  and 
Evaluation,  the  Office  of  the  Assistant  Secretary  for  Health, 
the  Office  of  the  Assistant  Secretary  for  Management  and 
Budget,  the  Health  Care  Financing  Administration,  and  the 
Office  of  Management  and  Budget. 
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evaluation  and  has  been  used  to  estimate  the  impact  of  PSRO  con- 
current review  on  days  of  care  (DOC)  and  discharges  (DISC)  per 
1000  Medicare  enrollees  and  average  length  of  stay  (ALOS).  Three 
developmental  studies  which  explore  additional  methodologies  for 
examining  utilization  data  were  also  conducted.  One  study,  the  20 
Percent  Sample  Medicare  Discharge  File  Study,  uses  diagnostic  and 
surgical  procedure  data  drawn  from  the  20  percent  Medicare  discharge 
file  for  the  years  1971  to  1973  and  1976  to  determine  if  the  PSRO 
program  impacted  on  selected  diagnoses  and  procedures.  The  two 
additional  utilization  analyses  are  based  primarily  on  available 
1976  and  1977  PSRO  Hospital  Discharge  Data  Sets  (PHDDS)  collected  by 
individual  PSROs.  The  MEDPAR-PHDDS/ 1  study  is  a  pre-post  analysis 
of  changes  in  ALOS  for  17  active  PSROs.  The  PHDDS  staging  study 
examines  changes  in  severity  of  illness  for  eight  active  PSROs.  The 
numbers  of  PSROs  studied  represent  the  universe  of  PSROs  with 
sufficient  PHDDS  data  to  support  ALOS  and  staging  analyses. 

1.2.2  Program  Costs 

The  19  78  evaluation  presents  PSRO  unit  costs,  analyzes  varia- 
tions in  delegated  review  unit  costs  and  estimates  costs.  For  the 
first  time,  PSRO  cost  data  were  obtained  from  the  PSRO  Management 
Information  System  (PMIS)/2  reports. 

Cost  projections  assumed  100  percent  review.  Although  current 
program  plans  call  for  implementation  of  focused  review  by  March 
19  79,  it  is  not  possible  to  estimate  precisely  the  extent  to  which 
focusing  and  other  budget-trimming  activities  can  reduce  these 
estimates  of  program  costs ./3  The  estimates  presented  here  provide 
a  rough  indication  of  the  decrease  in  costs  that  will  be  needed  for 
the  PSRO  program  to  operate  within  its  imposed  budget  constraints. 

1.2.3  Benefit-Cost  Analysis  of  Concurrent  Review 

The  benefits  from  concurrent  review  are  approximated  by  esti- 
mating from  the  100  percent  Medicare  claims  file  analyses  the  number 
of  "days  saved"  by  this  activity.  These  estimates  are  then  con- 
verted into  dollar  savings  through  the  use  of  a  simulation  process 


/I  MEDPAR  (Medicare  Provider  Analysis  and  Review)  is  a  series  of 
tables  developed  from  the  Medicare  20  percent  sample  discharge 
file  on  short-term  hospital  beneficiary  utilization.  The 
MEDPAR  data  are  used  here  for  baseline  purposes,  that  is,  to 
provide  data  for  the  period  when  the  PHDDS  was  not  collected. 

/2  The  PMIS  (PSRO  Management  Information  System)  contains  data 
submitted  by  the  individual  PSROs  to  generate  routine  reports 
on  review  activities,  review  costs,  PHDDS,  and  implementation 
status • 

/3  A  study  is  underway  to  determine  the  impact  of  focused  review 
on  program  costs.   Results  are  expected  in  the  fall  of  1979. 
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which  relates  program  utilization  impact  to  Medicare  program  expen- 
ditures. PSRO  concurrent  review  costs  are  related  to  the  dollar 
value  of  days  saved  by  the  benefit-cost  calculation. 

1.2.4  Medical  Care  Evaluation  (MCE)  Studies 

Two  evaluation  studies  of  MCE  activity  were  conducted.  The 
first  focused  on  descriptive  information  about  the  development  of 
the  MCE  program.  The  second  analyzed  the  results  from  a  pilot  study 
designed  to  assess  the  impact  of  MCEs.  The  technique  used  in  the 
study,  variation  rate  analysis,  measures  the  degree  to  which  MCE 
activity  reduced  variations  from  locally  established  standards  for 
medical  management  and  patient  outcomes  in  the  hospital  setting. 
The  estimates  of  change  in  variation  rates  presented  in  this  report 
are  highly  tentative  but  do  indicate  that  the  technique  is  useful 
and  can  shed  light  on  MCE  activity. /1_  Statistical  analyses  of 
variation  rates  are  supplemented  by  clinical  discussion  of  MCE 
perf  ormance . 

This  year's  MCE  analyses  do  not  link  directly  with  utilization 
or  benefit-cost  analyses.  The  1979  PSRO  evaluation  will  attempt  a 
benefit-cost  analysis  of  MCE  activity.  If  successful,  the  overall 
assessment  of  PSRO  benefits  and  costs  could  be  expanded  to  include 
both  CR  and  MCE. 

1.2.5  Program  Management 

Data  related  to  program  management  was  gathered  and  assessed 
to  provide  a  context  of  program  experience  within  which  program 
impact  could  be  measured  and  interpreted,  and  to  gain  insight  into 
possible  causal  relationships.  Health  Standards  and  Quality  Bureau 
(HSQB)  management  site  assessments  and  PSRO  regional  project  officer 
survey  data  were  reviewed  to  obtain  information  about  the  operating 
characteristics  of  PSROs  and  to  identify  those  aspects  of  the  PSRO 
program  which  appear  to  be  working  well. 

1.2.6  1978  Program  Status  and  Directions 

This  section  describes  the  status  of  PSRO  program  implementa- 
tion and  discusses  current  initiatives  intended  to  improve  the 
program's  performance. 

1.2.7  Directions  for  Future  Evaluation 

For  the  most  part,  this  section  serves  as  a  point  of  departure 
for  planning  the  1979  PSRO  evaluation.  The  1979  evaluation  will 
refine  1978  studies  and  place  more  emphasis  on  the  quality  of  care 
aspects  of  the  program.     Two  proposed  studies  are  the  MCE  variation 


/1      Results    from   a    larger    variation    rate   stu  cy   are    expected  by 
mid-1979. 
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rate  study  and  a  pilot  study  concerning  unnecessary  surgery. 
Results  from  demonstration  activities  in  ambulatory  and  long  term 
care  review  will  also  be  included. 

1.3    Evaluation  Issues 

This  subsection  discusses  issues  relevant  to  the  interpretation 
of  the  19  78  PSRO  evaluation.  First,  a  brief  comparison  of  the  1977 
and  19  78  evaluations  is  presented  and  followed  by  a  discussion  of 
the  implications  of  the  1978  evaluation  strategy.  The  section 
concludes  with  comments  which  relate  the  PSRO  program  to  the  broader 
regulatory  environment  in  which  it  operates  in  order  that  the 
magnitude  of  estimated  program  utilization  impact  can  be  placed  in 
an  overall  cost-containment  perspective. 

1.3.1  Comparison  With  1977  PSRO  Evaluation 

The  1978  evaluation  provides  a  more  comprehensive  analysis  than 
the  1977  evaluation.  Several  new  utilization  analyses  have  been 
added  to  provide  a  more  detailed  study  of  specific  aspects  of 
beneficiary  utilization  rates.  All  PSRO  areas  are  included  in  the 
100  percent  Medicare  claims  file  study  (so  there  are  more  active  and 
inactive  PSROs  under  study)  and  the  program  has  had  an  additional 
year  to  mature.  For  the  first  time,  cost  data  were  obtained  from 
the  PSRO  Management  Information  System  (PMIS)  with  the  result  that 
the  1978  cost  analyses  did  not  have  to  rely  solely  on  limited  sample 
data  as  did  the  1977  cost  studies.  The  estimation  procedures  which 
place  dollar  values  on  the  number  of  days  "saved"  by  the  PSRO 
program  are  more  precise  than  those  used  previously.  MCE  study 
impact  data  are  provided  for  the  first  time  and  program  experience 
data  are  more  rigorously  developed. 

1.3.2  Discussion  of  the  1978  Evaluation  Strategy 

Evaluation  Time  Frame;  Although  the  PSRO  program  has  insti- 
tuted several  major  management  initiatives  during  calendar  year 
1977,  data  for  the  ensuing  year  (1978)  are  generally  not  available 
for  assessing  program  impact.  In  essence,  then,  the  program  has 
progressed  one  year  beyond  the  performance  evaluations  discussed  in 
this  report.  Future  evaluations  will  be  required  to  analyze  the 
changes  instituted  in  the  program  during  1977  and  whose  full  measure 
may  not  yet  be  discernible  from  the  available  data. 

Data  Availability;  Estimates  of  utilization  impact  were 
limited  to  Medicare  beneficiaries  due  to  the  lack  of  national 
Medicaid  rate  data  aggregated  at  the  PSRO  level. /1    Because  aged 


/1  This  is  due  to  the  lack  of  uniform  estimates  of  the  Medicaid 
populations  by  state.  Without  these  estimates,  it  is  impos- 
sible to  calculate  Medicaid  beneficiary  use  rates  across  PSRO 
areas.  While  some  states  currently  have  Medicaid  use  and 
eligibility  data,  these  data  are  limited  in  terms  of  the  number 
of  PSROs  covered  and  are  not  comparable  to  the  years  of  Medi- 
care data  used  in  the  study. 
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Medicare  patients  represent  about  70  percent  of  all  Federal  dis- 
charges,/^ this  is  less  of  a  constraint  than  it  might  at  first 
appear.  On  the  other  hand,  any  attempt  to  generalize  Medicare 
patient  results  to  the  Medicaid  population  is  unwarranted. 

Further,  the  evaluation  does  not  try  to  measure  the  secondary 
utilization  impact  of  PSROs  on  non-Federal  patients.  MCE  impact 
analyses,  however,  do  relate  to  Federal  and  non-Federal  patients 
because  MCEs  study  all  patients. 

Program  Activities  Included  in  the  Evaluation  of  Program 
Impact ;  Estimates  of  program  impact  were  developed  for  both  con- 
current review  and  MCE  study  activities,  but  dollar  values  were 
assigned  only  to  changes  in  utilization  rates.  It  was  felt  that  the 
tentative  nature  of  the  MCE  impact  estimates  and  the  severe  methodo- 
logical difficulties  connected  with  assigning  dollar  values  to  MCE 
impact  did  not  warrant  extending  this  type  of  analysis  to  the  MCE 
studies  at  this  time. 

No  attempt  was  made  to  estimate  program  impact  for  ambulatory, 
long  term,  or  ancillary  care  services.  PSRO  program  activities  in 
these  areas  are  not  sufficiently  developed  at  this  time  to  support 
impact  evaluation.  Further,  due  to  the  data  limitations  discussed 
above,  the  evaluation  does  not  cover  activities  initiated  in  1978 
such  as  focused  review,  second  opinion  surgery  evaluation  programs 
(only  indirectly  related  to  the  PSRO  program)  and  efforts  to 
assist  intermediaries  in  the  screening  of  patient  bills. 

The  Potential  for  Self -Selection  Bias:  Approximately  half  of 
the  PSRO  areas  in  the  country  (96)  are  included  in  the  primary 
utilization  analysis.  If  the  96  active  and  93  non-active  PSRO  areas 
were  equivalent  in  all  ways  except  for  the  existence  of  PSRO  review, 
then  it  would  be  reasonable  to  assume  that  there  are  no  biases  in 
the  evaluation's  estimate  of  PSRO  review  impact  and  that  current 
inactive  areas  would  eventually  have  the  same  impact  on  utilization 
as  that  observed  for  the  active  areas.  In  reality,  however,  this 
assumption  must  be  carefully  considered. 

The  most  frequently  stated  argument  for  self-selection  bias 
asserts  that  early  funding  was  directed  towards  those  PSRO  areas 
which  possessed  the  greatest  desire  and  willingness  to  take  action. 
The  counter  argument  is  that  early  PSROs  might  represent  a  bias 
against  change  inasmuch  as  the  medical  profession  viewed  the  PSRO 
program  as  an  opportunity  to  control  the  review  process  and,  there- 
fore, to  preserve  the  status  quo.  Observers  of  the  PSRO  program 
have  vehemently  argued  both  sides  of  the  question  and  it  is  not 
obvious   to   the   evaluation   team  which   contention  is  more  correct. 


/1  Aged  Medicare  patients  represent  an  even  higher  proportion  of 
Federal  paid  days  of  care  due  to  the  relatively  higher  average 
lengths  of  stay  associated  with  elderly  Medicare  beneficiaries. 
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The  1978  evaluation  defines  as  active  all  PSROs  with  at 
least  one  hospital  under  review  as  of  the  first  quarter  of  1977. 
While  this  definition  places  relatively  new  PSROs  in  the  active 
category,  the  estimates  of  PSRO  utilization  impact  were  not  affected 
by  this  classification  nearly  as  much  as  might  have  been  antici- 
pated* New  PSR0s/1  were  analyzed  separately  and  found  to  be  nearly 
as  effective  as  PSROs  that  had  been  operational  for  over  a  year. 
PSROs  that  were  just  starting  out  in  1977  did  not  appear  to  be 
markedly  less  effective  than  those  PSROs  which  had  been  functioning 
longer. 

As  shown  in  Table  3,  the  study's  96  active  PSROs  are  dis- 
tributed fairly  evenly  throughout  the  United  States'  four  major 
census  regions. 

TABLE  3 

DISTRIBUTION  OF  ACTIVE  PSROS  BY  CENSUS  REGION 
(March  1977) 


Northeast 

North  Central 

South 

West 

Total 

Active 

32 

21 

19 

24 

96 

Inactive 

14 

30 

38 

1 1 

93 

Total 

46 

51 

57 

35 

189 

Percentage 
Active 

70% 

41% 

33% 

69% 

51% 

However,  the  regression  results  indicate  that  population 
density  is  correlated  with  program  impact,  and  the  non-active  PSRO 
areas  are  much  less  densely  populated  than  the  active  areas. 
Furthermore,  the  North  Central  and  Southern  regions  have  the  least 
number  of  active  PSROs  (41  and  33  percent  of  designated  areas 
respectively),  and  are  associated  with  the  lowest  estimated  PSRO 
utilization  impact. 

This  discussion  suggests  that  the  future  expansion  of  the 
PSRO  program  will  yield  a  variety  of  results  not  unlike  those 
already  experienced.  On  balance,  it  is  nearly  impossible  to  predict 
the  degree  to  which  new  PSROs  will  perform  better  or  worse  than  the 
96  PSRO  areas  examined  herein. 


/1    There  were  31  active  PSROs  with  less  than  50  percent  of  hospital 
admissions  under  review  as  of  April  1,  1977. 
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There  are  two  reasons,  however,  to  expect  that  future  evalua- 
tions of  the  program's  impact  on  hospital  expenditures  will  show 
improvements.  The  first  is  that  the  program  has  responded  to 
pressures  for  cost-effectiveness  by  instituting  changes  in  review 
procedures  and  negotiating  objectives  with  the  PSROs.  The  first 
results  of  these  changes  may  be  evident  in  the  1979  evaluation.  A 
second  reason  for  optimism  is  that  the  potential  effect  of  the 
program  on  utilization  is  large  relative  to  the  effect  estimated  in 
the  present  evaluation  (1.5%).  Estimates  of  unnecessary  admissions, 
services,  procedures,  and  days  of  care  vary  greatly  due  to  the  early 
stage  of  methodological  development  in  this  area.  No  matter  how  the 
estimates  are  generated,  however,  they  indicate  a  great  deal  more 
unnecessary  care  —  10  to  20  percent  —  than  has  thus  far  been 
influenced  by  PSRO.  The  potential  program  impact  is  thus  much 
greater  than  its  performance  as  measured  by  1977  data.  Management 
initiatives  may  begin  to  realize  this  potential. 

Incremental  Nature  of  PSRO  Utilization  Benefits;  The  estimates 
of  utilization  impact  developed  in  the  1978  report  are  "incremental 
benefits"  in  that  active  PSRO  areas  are  compared  to  non-active  PSRO 
areas,  where  some  form  of  hospital-based  utilization  review  (UR)  was 
generally  being  carried  out . / 1  Thus,  the  evaluation  study  measures 
the  difference  between  the  additional  effect  of  PSRO  concurrent 
review  and  the  impact  of  non-PSRO,  hospital-based  UR.  To  the  extent 
that  enactment  of  PSRO  legislation  induced  more  effective  hospital- 
based  UR  in  non-PSRO  areas  and,  as  such,  had  an  indirect  impact  on 
UR  overall,  the  19  78  evaluation  underestimates  the  impact  of  the 
PSRO  legislation  on  constraining  hospital  utilization.  While 
previous  empirical  work  has  suggested  that  such  UR  programs  have  had 
little  or  no  impact,  the  degree  to  which  the  study's  incremental 
benefits  underestimate  the  full  impact  of  PSRO  concurrent  review  is 
essentially  unknown. 

The  primary  implication  of  the  "incremental  benefit"  approach 
to  estimating  utilization  impact  relates  to  the  fact  that  the 
study's  benefit-cost  results  are  obtained  by  comparing  incremental 
benefits  with  the  full  costs  of  concurrent  review.     To  the  extent 


/1  Utilization  review  (UR)  was  required  for  Medicare  participation 
by  the  original  Medicare  Law  (P.L.  89-97)  and  for  Medicaid  by 
the  1967  Social  Security  Amendments  (P.L.  90-248).  The  origi- 
nal UR  regulations  were  strengthened  considerably  in  1974 
(Federal  Register,   39FR41604,  November  29,  1974). 
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that  benefits  are  conservatively  estimated,  this  approach  under- 
states the  benefit-cost  ratio  estimated  for  the  program. /1 


/I  Benefit-cost  (BC)  techniques  frequently  relate  full  costs  to 
full  benefits.  A  hypothetical  numerical  example  is  presented 
which  compares  this  study's  BC  ratio,  which  uses  incremental 
benefits  and  full  costs,  with  the  usual  BC  measure.  This 
study's  BC  measure  is  also  compared  to  a  BC  measure  based 
on  incremental  benefits  and  incremental  costs.  The  latter  BC 
measure  was  used  in  the  1977  PSRO  evaluation. 

For  illustrative  purposes  assume  benefits  and  costs  are  as 
follows : 

Benefits  Costs 

UR  2  10 

PSRO  15  12 

Incremental  Incremental 
PSRO  Impact  13      PSRO  Costs  2 

As  stated  above,  the  BC  technique  used  in  the  1978  evaluation 
compares  full  PSRO  costs  (12)  to  incremental  benefits  (13), 
yielding  a  BC  ratio  of  13:12  which  is  greater  than  1:1.  If 
full  PSRO  costs  were  compared  to  full  benefits,  the  hypothe- 
tical BC  ratio  would  be  slightly  greater,  namely,  15:12.  The 
BC  measure  used  in  the  1978  evaluation  is  approximately  correct 
when  UR  benefits  are  close  to  zero.  While  this  assumption 
was  not  tested  in  the  1978  evaluation,  previous  studies  have 
suggested  that  this  is  indeed  the  case.  Accordingly,  it  is 
very  likely  that  the  study  BC  measure  is  a  close  approximation 
of  the  BC  measure  most  frequently  used. 

However,  BC  measures  based  on  comarisons  of  incremental  PSRO 
benefits  to  incremental  PSRO  costs  can  be  misleading.  The 
numbers  above,  for  instance,  yield  a  full  BC  measure  of  15:12 
while  the  incremental  BC  measure  is  13:2,  highly  dissimilar 
ratios.  It  is  also  possible  for  the  incremental  PSRO  BC  ratio 
to  be  greater  than  1:1  and  yet  have  a  full  PSRO  BC  ratio  less 
than  1:1.  For  example,  if  in  the  above  set  of  hypothetical 
numbers  PSRO  benefits  were  10  as  opposed  to  15,  the  full  PSRO 
BC  ratio  would  be  10:12  which  is  less  than  1:1.  But,  with  this 
example,  the  incremental  BC  ratio  would  be  8:2  which  is  con- 
siderably greater  than  1:1.  Thus  a  highly  favorable  incre- 
mental BC  ratio  can  mask  an  unfavorable  program  BC.  For  this 
reason  the  1978  PSRO  evaluation  does  not  replicate  the  1977  BC 
technique • 
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1.3.3     The  Relationship  Between  PSROs  and  Cost  Containment 

Health  care  expenditures  are  affected  both  by  the  price  paid 
for  health  care  services  and  the  quantity  and  composition  of  ser- 
vices consumed.  Many  factors  influence  these  expenditures/  for 
example,  increases  in  numbers  and  staff  levels  of  personnel,  and 
increases  in  DOC  and  services  per  day.  Of  these  factors,  however, 
the  PSRO  program  has  a  legislative  mandate  only  to  control  utiliza- 
tion, namely  days  of  care  and  volume  of  services.  Currently  the 
program  focuses  primarily  on  DOC.  While  estimates  of  the  relative 
influence  of  causal  factors  vary,  it  is  generally  agreed  that  in 
recent  years  changes  in  hospital  days  of  care  have  represented  only 
a  secondary  influence  on  the  rate  of  increase  in  hospital  expendi- 
tures. /1-5 

Thus  far,  as  PSROs  have  directed  their  energies  primarily 
towards  the  reduction  of  hospital  utilization  rates,  there  have  been 
only  limited  reductions  in  staffing  and  fixed  costs.  Because  more 
than  half  of  a  hospital  per  diem  is  related  to  fixed  costs,  a 
reduction  in  a  day  of  care  merely  distributes  these  fixed  costs  to 
remaining  days  of  care,  i.e.,  declines  in  patient  volume  lead  to 
increases  in  hospital  unit  prices  in  order  to  maintain  revenues 
equal  to  expenditures.  From  the  perspective  of  the  Medicare  and 
Medicaid  programs,  hospital  days  saved  translate  into  much  smaller 
declines  in  reimbursed  costs  than  would  be  the  case  if  a  higher 
proportion  of  hospital  costs  were  variable  rather  than  fixed. 

Since  a  reduction  in  days  of  care  has  a  relatively  small  impact 
on  hospital  costs,  the  current  PSRO  program  would  have  to  reduce 
days  of  care  by  22  percent  in  order  to  reduce  Medicare  hospital  care 
expenditures  by  11  percent,  which  is  approximately  the  current 
annual  rate  of  increase  in  Medicare  hospital  expenditures.  This 


/1  Congressional  Budget  Office,  Expenditures  for  Health  Care, 
Federal   Programs  and  Their  Effects   (August  19  77) ,   p.  28. 

/2  Council  on  Wage  and  Price  Stability,  "Labor -Management  Inno- 
vations in  Controlling  Cost,"  Part  III,  Federal  Register, 
(Sept.   17,   1976,  pp.  40309  and  40311). 

/3  Council  on  Wage  and  Price  Stability,  Executive  Office  of 
the  President,  "The  Rise  of  Hospital  Costs,"  (January  1977), 
p.  iii. 

/4  American  Hospital  Association,  Hospital  Statistics,  1976 
Edition  from  1975  Annual  Survey,  p.  XIV. 

/5  Gibson,  R.M.  and  Fisher,  R.C.,  "National  Health  Expenditures, 
Fiscal   Year    1977,"    Social   Security  Bulletin,   July   19  78. 
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would  translate  into  a  program  benefit-cost  ratio  of  15:1.  Such  a 
ratio  exceeds  returns  found  for  either  other  government  programs  or 
the  private  sector.  It  is  unreasonable  to  expect  the  PSRO  program 
alone  to  be  able  to  control  Federal  health  costs. 

The  PSRO  program's  impact  on  health  care  expenditures  may  be 
augmented  as  PSROs  initiate  review  of  ancillary  services,  implement 
profile  analyses  and  interact  more  closely  with  health  planning 
agencies.  Given  the  variety  of  factors  which  influence  health 
expenditures,  a  comprehensive  cost  containment  program  is  required 
where  each  component  devotes  its  energies  to  a  particular  aspect  of 
the  cost-containment  strategy.  If  any  one  program  fails,  the 
broader  strategy  is  weakened  because  numerous  factors  work  indepen- 
dently to  affect  prices  and  quantity  of  health  care  services. 

1.4    Summary  of  Evaluation  Results 

This  subsection  highlights  evaluation  findings.  Individual 
study  results  are  presented  as  follows:  Utilization  Impact,  Program 
Costs,  Benefit-Cost  Analysis  of  Concurrent  Review,  Medical  Care 
Evaluation  Studies,  Program  Management,  1978  Program  Status  and 
Directions,  and  Directions  for  Future  Evaluation. 

1.4.1    Utilization  Impact 

The  Evaluation's  four  utilization  studies  are  discussed  below 
in  the  following  order:  ( 1)  100  Percent  Medicare  Claims  File  Study; 
(2)  20  Percent  Sample  Medicare  Discharge  File  Study;  (3)  MEDPAR- 
PHDDS  Study;  and  (4)  PHDDS  Staging  Study. 

1.4.1.1     100  Percent  Medicare  Claims  File  Study 

This  is  the  most  comprehensive  study  of  the  impact  of  PSROs 
undertaken  by  the  1978  PSRO  evaluation.  This  study  estimates  the 
impact  of  concurrent  review  on  acute  hospital  inpatient  use.  It  is 
similar  in  approach  to  the  100  Percent  Medicare  Utilization  Study 
included  in  the  1977  PSRO  evaluation  in  that  multiple  regression 
techniques/ 1  are  used  to  determine  the  effect  of  PSRO  concurrent 
review  on  days  of  care  per  1,000  Medicare  enrollees  (DOC),  dis- 
charges per  1,000  Medicare  enrollees  (DISC),  and  average  lengths  of 
stay  (ALOS).  Tests  are  performed  to  determine  if  PSROs  are  affect- 
ing the  proportion  of  short  and  long  stays  (less  than  four  days  and 
more  than  21  days  respectively).  The  primary  variable  of  interest 
is  DOC  because  of  this  variable's  implications  for  Medicare  program 
expenditure  levels.  DISC  and  ALOS  are  used  to  explain  why  DOC 
varies  (that  is,  days  of  care  are  determined  by  discharge  rates  and 
ALOS). 

The  study  is  based  on  a  comparative  analysis  wherein  all  96 
areas  with  active  PSROs  (i.e.,  PSROs  with  at  least  one  hospital 
under  review  as  of  the  first  quarter  of  1977)  are  compared  to  all  93 


/1  Forced  order  regressions  were  used  so  that  the  test  of  signifi- 
cance for  the  program  effect  was  able  to  be  done  after  the 
inclusion  of  the  other  explanatory  variables. 
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areas  without  active  PSROs  for  the  period  1974  to  1977.  No  attempt 
was  made  to  select  matched  comparison  areas,  thereby  eliminating  the 
possibility  of  biasing  the  results  by  subjective  selection  of 
matched  test  and  comparison  groups.  PSRO  program  effects  are 
estimated  by  comparing  the  relative  rates  of  change  in  hospital 
utilization  between  active  and  inactive  PSRO  areas.  If  hospital 
utilization  rates  fall  more  rapidly  (or  rise  less  rapidly)  in  PSRO 
areas  after  PSRO  implementation,  the  PSRO  program  is  credited  with 
utilization  impact. 

Two  types  of  analyses  are  presented:  first,  impact  for  the 
program  as  a  whole  is  estimated  and  second,  individual  PSROs  are 
ranked  according  to  the  magnitude  of  the  individual  impact  estimated 
for  each  active  PSRO.  The  latter  set  of  estimates  enables  identi- 
fication of  "successful"  PSROs.  All  analyses  are  statistically 
adjusted  to  account  for  the  effects  on  hospital  utilization  of 
numerous  non-PSRO  factors  (i.e.,  number  of  SNF  beds/1  , 000,  etc.) 
which  are  thought  to  affect  hospital  utilization  and  changes  in 
hospital  utilization.  Such  adjustments  help  to  ensure  that  esti- 
mates of  program  impact  are  not  confounded  by  the  impact  of  other 
factors  which  might  influence  utilization. 

Before  presenting  the  regression  model  and  a  summary  of  regres- 
sion results,  the  study's  primary  data  are  described.  The  dependent 
variables  DOC,  DISC,  and  ALOS  are  presented  in  Table  4  and  Figure  1 
for  each  six-month  period  for  which  data  were  available.  These  data 
illustrate  a  number  of  interesting  facts.  First,  rates  in  both 
active  and  inactive  PSROs  have  followed  a  similar  pattern  over  time. 
Second,  days  of  care  (DOC)  have  not  changed  appreciably  between  1974 
and  19  77;  the  discharge  rate  has  steadily  risen  and  the  average 
length  of  stay  has  decreased. 

There  is  a  distinct  cyclical  pattern  in  both  DOC  and  DISC.  In 
the  first  half  of  each  year,  DOC  are  from  40  to  140  days  higher  than 
in  the  second  half  of  eacn  year.  This  cyclical  pattern  is  typical 
of  medical  care  use.  In  the  colder  months  in  the  first  half  of  the 
year  there  is  more  sickness.  In  addition,  the  cold  months  in  the 
second  half  of  the  year  (November  and  December)  are  holiday  times 
when  discretionary  hospitalization  may  be  postponed. 

This  pattern  appears  to  be  related  to  PSRO  performance.  The 
regression  analyses  show  that  the  PSRO  impact  on  utilization  is 
almost  entirely  in  the  first  half  of  the  year.  This  may  be  because 
occupancy  tends  to  be  higher  in  the  first  half  of  the  year,  generat- 
ing pressures  towards  a  rapid  turnover  of  patients  to  permit  new 
patients  to  be  admitted.  When  occupancy  rates  are  lower,  in  the 
summer,  fall,  and  early  winter,  there  may  be  a  tendency  to  keep 
patients  hospitalized  longer  so  as  to  maintain  revenues.  Thus,  PSRO 
activities  would  be  expected  to  be  more  successful  during  the 
periods  of  high  occupancy.  The  relationship  between  PSRO  effect  and 
cyclical  use  suggests  that  occupancy  may  even  be  a  controlling 
factor. 
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FIGURE  1 :  Medicare  Total  Days  of  Care  per  1,000  Aged  Enrol  lees. 
Medicare  Discharges  per  1,000  Aged  Enrollees  and 
Average  Length  of  Stay  for  Active  and  Inactive  PSROs 


Active  PSRO  Areas  (n=96) 
Inactive  PSRO  Areas  (n=93) 
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A  third  point  is  that  active  and  inactive  areas  have  use  rates 
that  are  quite  similar  to  one  another.  Days  of  care  are  only  one  or 
two  percentage  points  lower  in  active  areas,  and  discharges  tend  to 
be  five  or  six  percentage  points  lower,  whereas  ALOS  is  higher  in 
active  areas  by  about  six  tenths  of  a  day  per  discharge. 

Finally,  there  has  been  very  little  change  in  the  difference 
between  active  and  inactive  PSRO  areas  over  time.  As  shown  in 
Figure  1 ,  the  relative  relationships  between  active  and  inactive 
PSROs  have  hardly  changed.  The  actual  differences  are  also  small 
and  are  exaggerated  in  Figure  1  since  most  of  the  space  for  each 
table  has  been  truncated  (i.e.,  the  vertical  axes  do  not  run  from 
zero).  Hence,  the  impact  that  can  be  demonstrated  for  PSRO  activity 
on  utilization  rates  in  the  regression  analysis  is  likely  to  be 
small. 

The  regression  models  relate  the  dependent  variables  presented 
above  to  a  series  of  independent  variables: 

•  baseline  ( 1974  pre-PSRO)  utilization  levels ,/1 

•  proportion  of  total  population  age  65  or  over, 

•  short-stay  hospital  beds/1, 000  population, 

•  population  density, 

•  proportion  of  total  hospital  days  due  to 
Medicare  enrollees, 

•  physicians/1,000  population, 

•  nursing  home  beds/1, 000  population, 

•  occupancy  rate, 

•  proportion  of  families  with  incomes  under 
$5,000  and 

•  PSRO  program  variable. 

Three  variables,  the  proportion  of  aged  population,  proportion 
of  hospital  days  due  to  Medicare,  and  nursing  home  beds/1, 000 
population,  are  used  in  the  1978  evaluation  for  the  first  time.  The 
other  independent  variables  were  used  in  the  1977  report. 

The  regression  model  relates  each  dependent  variable  to  the 
independent  variables,  thereby  determining  the  impact  of  the  PSRO 
program  adjusted  for  variations  in  the  remaining  independent  vari- 
ables. For  instance,  the  inclusion  of  the  1974  baseline  DOC  levels 
adjusts  PSRO  program  impact  estimates  for  variations  in  pre-PSRO  DOC 
levels.  The  regression  results  thereby  reflect  the  relative  effec- 
tiveness of  the  96  active  PSROs  as  compared  to  the  93  inactive  PSRO 
areas. 


/1      Each  dependent  variable  uses  its  own  unique  baseline,  i.e.,  DOC 
uses  1974  DOC,  DISC  uses  1974  DISC,  etc. 
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A  variety  of  regression  model  formulations  were  estimated  with 
highly  similar  results.  The  results  presented  below  are  those  which 
the  evaluation  team  felt  were  most  appropriate. 

The  primary  regression  analyses  found  a  significant  effect  for 
DOC,  DISC,  but  not  for  ALOS.  The  estimate  of  reduction  in  DOC 
attributable  to  PSRO  concurrent  review  was  1.5  percent.  This 
effect  was  predominantly  in  the  first  half  of  the  year.  No  effect 
was  found  for  the  second  half  of  the  year./2  The  average  reduction 
in  DOC  attributable  to  PSRO  for  the  entire  year  was,  however, 
significant.  There  was  no  PSRO  effect  on  long-stay  or  short-stay 
discharges  in  any  of  the  regression  analyses. 

The  above  are  average  or  aggregate  results  reflecting  overall 
program  performance.  Disaggregated  analyses  were  also  performed 
which  associate  impact  with  any  given  PSRO  or  group  of  PSROs.  All 
but  15  of  the  96  active  PSROs  showed  some  decrease  in  the  days  of 
care  per  1,000  aged  Medicare  enrollees.  The  "best"  PSRO  showed  a 
decrease  of  8.75  percent,  whereas  the  "worst"  PSRO  showed  an  in- 
crease of  1.96  percent. 

A  breakdown  of  PSRO  impact  by  region  of  the  country  showed  that 
PSROs  in  the  Northeast  and  the  West  had  the  greatest  impact  on  DOC 
with  reductions  of  2.7  percent  and  2.6  percent,  respectively.  PSROs 
in  the  North  Central  part  of  the  country  showed  a  small  impact  (0.5 
percent  reduction)  and  PSROs  in  the  South  had,  on  average,  no  effect 
on  DOC.  When  PSRO  impact  is  examined  in  terms  of  the  age  of  the 
PSRO,  the  analysis  suggests  that  older  PSROs  may  be  more  effective 
than  newer  ones.  The  hypothesis  that  PSRO  effectiveness  increases 
with  age  could  not  be  substantiated,  however,  through  multivariate 
(as  opposed  to  descriptive)  analysis. 

An  additional  regional  analysis  was  performed  by  entering 
regions  explicitly  into  the  regression  in  dummy  variable  ("0"  or 
" 1")  form.  This  procedure  resulted  in  four  separate  active/inactive 
comparisons  (one  for  each  region  of  the  country).  The  difference 
between  this  procedure  and  that  summarized  above  is  that  active  and 
inactive  PSROs  are  effectively  compared  within  each  region  rather 
than  nationally.  Thus,  active  PSROs  in  the  Northeast  reduced  DOC 
4.0  percent  relative  to  inactive  PSROs  in  the  Northeast.  Similarly, 
active  PSROs  in  the  North  Central  and  West  regions  of  the  country 
reduced  DOC  by  1.6  percent  and  2.0  percent  relative  to  inactive 
PSROs  in  these  regions.  Active  PSROs  in  the  South  increased  DOC  by 
1.3  percent  relative  to  inactive  PSROs  in  the  South.  These  regional 
estimates,  while  slightly  different  from  those  presented  above, 
indicate  the  same  ordering  of  regional  impact.  The  entering  of 
regional  variables  into  the  regression  model  did  not  materially 
affect  the  aggregate  estimate  of  PSRO  impact. 


/1      This   result   is   consistent  with  the  findings   of   the   1977  PSRO 
evaluation. 
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The  existence  of  hospital  rate  review  activities  in  PSRO 
areas  could  also  have  affected  PSRO  impact.  To  test  for  this 
possibility,  a  hospital  rate  review  dummy  variable  was  entered  in 
the  regression  equation.  Analysis  of  results  indicated  that  hos- 
pital rate  review  activities  do  not  change  PSRO  impact  or  interact 
with  PSRO  review  activities.  However,  hospital  rate  review  does 
affect  DOC  independently  of  PSRO  review.  Rate  review  increases  DOC 
by  approximately  3.5  percent. 

In  addition,  an  effort  was  made  to  determine  the  degree  to 
which  patients  certified  for  care  at  skilled  nursing  facilities 
(SNFs)  were  receiving  care  in  acute  care  settings  due  to  the  un- 
availability of  Medicare-certified  SNF  beds.  Data  were  collected 
from  19  PSROs  which  responded  to  a  request  for  information . /1 
Approximately  1.9  percent  of  all  Medicare  acute-care  inpatient  days 
in  these  19  PSRO  areas  were  actually  SNF-certif ied,  meaning  that 
their  care  should  have  been  provided  in  an  SNF  instead  of  an  acute- 
care  setting,  if  SNF  beds  were  available. 

This  compares  to  the  overall  PSRO  DOC  impact  estimate  of  1.5 
percent.  The  degree  to  which  the  1.9  percent  figure  represents  a 
potential  for  PSRO  impact  is  uncertain.  If  the  availability  of  SNF 
beds  were  increased  in  both  active  and  inactive  PSROs  there  could 
well  be  a  uniform  decrease  in  acute  days  of  care.  If  this  were  the 
case,  then  the  impact  would  be  due  to  SNF  bed  availability,  not  PSRO 
review,  although  PSROs  could  presumably  take  credit  for  having 
encouraged  this  kind  of  substitution. 

1.4.1.2  20   Percent  Sample  Medicare  Discharge  File  Study 

The  2  0  Percent  Sample  Medicare  Discharge  File  Study  (the  first 
of  the  three  developmental  studies  presented  in  this  subsection), 
attempts  to  refine  results  produced  by  the  1977  PSRO  evaluation  and 
to  develop  methods  applicable  to  future  data  bases.  In  particular, 
it  was  felt  that  even  though  PSROs  had  no  impact  on  total  utiliza- 
tion rates  in  19  76,  as  shown  in  the  1977  Evaluation  Report,  there 
was  still  a  possibility  that  impact  on  certain  diagnostic  or  pro- 
cedural categories  could  be  demonstrated. 

A  number  of  surgical  procedures  and  diagnostic  groups  were 
selected  for  study  for  which  there  was  reason  to  believe  that 
decreases  in  utilization  might  be  appropriate.  A  priori  reasons  for 
believing  that  changes  might  be  expected  to  occur  were  determined 
from  a  review  of  the  medical  literature  and  from  a  list  of  potential 


/1  These  19  PSROs  were  selected  in  terms  of  their  willingness  to 
cooperate  and  their  availability  of  data;  by  no  means  do  they 
represent  a  statistically  valid  sample. 
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PSRO  impact  areas  given  in  the  1975  PSRO  Evaluation  Plan./J^  In 
addition,  a  number  of  other  surgical  and  diagnostic  categories  were 
included  in  the  analysis  as  an  exploration  of  potential  PSRO  impact 
areas.  The  total  number  of  surgical  procedure  and  diagnostic 
categories  examined  was  42. 

Active  PSRO  areas  were  compared  with  inactive  PSRO  areas  to 
examine  changes  in  use  rates  between  the  pre-PSRO  period  ( 1971 
through  1973)  and  the  post-implementation  period  (1976).  The 
analysis  centered  on  high-use  geographic  areas,  although  average-use 
and  low-use  geographic  areas  were  also  examined. 

Regarding  the  analysis  to  date,  the  results  appear  similar  to 
those  of  the  19  77  Medicare  utilization  study.  Little  PSRO  effect, 
was  shown,  although  there  were  signs  that  PSROs  might  have  had 
impact  in  certain  diagnostic  and  procedural  areas.  In  particular, 
some  categories  with  high  use  may  be  areas  where  PSROs  show  some 
improvement.  The  continuation  of  this  study  with  1977  data  should 
permit  a  more  thorough  examination  of  program  impact  on  selected 
diagnoses. 

1.4.1.3    MEDPAR/PHDDS  Study 

The  MEDPAR-PHDDS  study  is  an  analysis  of  changes  in  inpatient 
ALOS  by  Medicare  enrollees  after  implementation  of  PSRO  review. 
Because  PHDDS  is  available  only  after  a  PSRO  has  become  implemented, 
prior  ALOS  data  were  taken  from  the  MEDPAR  data  maintained  by 
Medicare.  Due  to  the  obvious  problems  involved  in  using  MEDPAR  for 
pre-period  and  PHDDS  for  post-period  observations,  a  study  was 
performed  to  determine  the  comparability  of  MEDPAR  and  PHDDS  data. 

The  comparison  study  uses  1976  estimates  from  PHDDS  and  MEDPAR 
for  the  seven  PSROs  which  had  1976  PHDDS  data. 

The  results  of  the  comparison  study  indicate  that  aggregate 
ALOS  estimates  are  comparable  for  the  two  data  sets  and  that  finer 
breakdowns  are  still  comparable  as  long  as  the  disaggregations  are 
on  a  well-defined  variable  such  as  age.  However,  data  developed  by 
processes  involving  judgment  may  vary  greatly;  thus,  diagnostic  and 
procedural  data  are  not  comparable  between  the  two  data  sets. 

A  pre-post  analysis  was  subsequently  performed  on  those  data 
elements  for  which  MEDPAR  and  PHDDS  seemed  comparable.  The  analysis 
failed  to  demonstrate  a  statistically  significant  PSRO  effect  on 
utilization  trends. 


/1  DHEW,  Office  of  the  Assistant  Secretary  for  Health,  Office  of 
Professional  Standards  Review,  Program  Evaluation  Plan; 
Professional  Standards  Review  Organization,  September  22, 
1975. 
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This  MEDPAR-PHDDS  study  was  a  first  attempt  to  use  PHDDS  for 
evaluative  purposes.  The  value  of  this  data  set  will  be  in  future 
studies  of  relative  PSRO  performance  across  PSROs.  Because  PHDDS 
contains  Medicaid  and  Maternal  and  Child  Health  discharges  as  well 
as  Medicare,  relative  PSRO  effect  on  these  beneficiaries  can  be 
evaluated  as  well./1 

1.4.1.4    PHDDS  Staging  Study 

Staging  is  a  process  by  which  inpatient  stays  within  diagnostic 
groups  can  be  grouped  according  to  criteria  of  severity.  Three 
stages  of  severity  are  used:  stage  1  comprises  the  least  severe 
cases;  stage  2,  more  severe;  and  stage  3,  the  most  severe  cases. 
The  1978  staging  study  applied  staging  algorithms  to  the  PHDDS.  One 
objective  was  to  test  whether  or  not  PSRO  review  had  any  impact  on 
the  case  severity  within  diagnostic  discharge  categories.  A  second 
objective  was  to  determine  if  there  had  been  changes  in  ALOS  within 
stages  of  diagnostic  categories. 

A  pilot  test  applying  the  staging  methodology  to  a  sample  of 
the  PHDDS  suggested  that  there  was  sufficient  information  on  the 
abstracts  to  perform  the  staging  analysis ./2_  However,  comparison 
of  staging  results  using  PHDDS  abstracts  and  CPHA/3  abstracts  from 
the  same  hospitals  yielded  a  consistently  higher  level  of  severity 
with  CPHA  abstracts  than  with  PHDDS  abstracts.  This  suggests  that 
PHDDS  may  not  be  selecting  the  secondary  diagnoses  which  are  being 
picked  up  through  the  CPHA  abstracting  process.  Thus,  while  PHDDS 
may  support  staging  analyses,  it  may  not  provide  results  comparable 
to  those  obtained  from  the  staging  of  CPHA  abstracts.  Staging 
results  should  not  be  compared  across  the  two  data  sets. 

Notwithstanding  the  potential  problems  of  the  PHDDS  abstracting 
process,  analyses  were  performed  to  explore  for  changes  in  case 
severity  and  ALOS  in  a  small  number  of  active  PSROs.  The  results 
indicate  that  in  six  diagnostic  areas  there  may  be  a  trend  towards  a 
reduction  in  the  percentage  of  stage  one  admissions,  suggesting  that 
there  may  be  fewer  inappropriate  hospital  admissions.  However,  in 
one  instance,  carcinoma  of  the  breast,  stage  one  admissions  rose  as 
a  percentage  of  total  breast  carcinoma  admissions,  which  may  repre- 
sent a  positive  trend  toward  earlier  treatment  of  this  disease. 


/1  Since  PHDDS  does  not  contain  the  number  of  eligibles  for  the 
Medicaid  population,  Medicaid  rate  data  cannot  be  generated. 

/2  A.  Stonecipher  et  al . ,  "Some  Preliminary  Results  from  Staging 
Analyses  of  the  PSRO  Hospital  Discharge  Data  Set,"  Unpublished 
report,  ( SysteMetrics  Inc.,  June  1978). 

/3      Commission  on  Professional  and  Hospital  Activities. 


25 


ALOS  analyses  showed  that  there  were  some  instances  of  declines  in 
ALOS  in  all  three  stages.  An  analysis  of  potentially  confounding 
variables  showed  that  long-term  care  beds/1 ,000  aged  persons  and 
physicians/1/000  persons  influence  length  of  stay  independently  of 
PSRO  review  and  therefore  may  override  any  PSRO  impact. 

The  staging  study  results  must  be  considered  as  highly  tenta- 
tive. The  increase  in  severity  level  may  be  entirely  an  artifact  of 
the  data  in  that  more  accurate  reporting  would  indicate  a  more 
severe  case  mix.  A  validation  study  is  currently  underway  to 
determine  the  degree  to  which  data  recording  procedures,  as  opposed 
to  PSRO  review,  explain  observed  results. 

1.4.2     Cost  Analyses 

Several  analyses  of  PSRO  review  costs  were  undertaken.  The 
objectives  of  these  studies  were  to  examine  unit  costs  and  varia- 
tions in  program  expenditures,  identify  systematic  correlations 
between  delegated  hospital  costs  and  hospital  characteristics,  and 
project  the  costs  of  a  fully  implemented  program  as  of  1977.  As 
stated  above,  no  attempt  was  made  to  estimate  PSRO  costs  over  and 
above  those  of  hospital-based  utilization  review.  Thus,  coats 
presented  here  represent  the  full  costs  of  any  PSRO  component  under 
consideration. 

The  primary  data  sources  for  these  studies  are  Health  Standards 
and  Quality  Bureau  (HSQB)  Forms  151  and  153.  Although  a  study  is 
currently  underway  to  assess  the  validity  of  these  data,  use  of  the 
non-validated  data — necessary  for  completion  of  these  cost  studies — 
introduces  an  unknown  element  of  variation  in  the  analysis.  For 
example,  to  the  extent  that  there  is  greater  underreporting  of 
discharges  reviewed  than  program  expenditures,  unit  cost  estimates 
will  be  biased  upward.  The  most  recent  data  reflect  FY  1977  program 
experience  and  do  not  reflect  any  cost  reductions  which  might  result 
from  the  use  of  focused  review  or  the  increased  cost  consciousness 
of  program  managers  following  the  1977  PSRO  Evaluation.  Cost  data 
have  been  adjusted  to  reflect  calendar  year  1977  experience  in  order 
to  evaluate  calendar  year  1977  benefit  estimates. 

Program  cost  data  submitted  to  HSQB  by  individual  PSROs  were 
examined  to  assess  gross  variations  in  unit  costs  for  FY  1977.  The 
primary  focus  of  this  assessment  was  on  unit  expenditures  for  total 
hospital  review,  i.e.,  concurrent  review,  medical  care  evaluation 
studies,  and  areawide  review.  Management  and  support  costs  were 
also  examined,  but  for  this  analysis  they  were  not  allocated  to  the 
individual  components  within  total  hospital  review.  Expenditures 
for  long-term  care  or  ambulatory  review  also  were  not  included. 
Unit  costs  were  computed  by  dividing  the  relevant  annual  expen- 
ditures   by    the    number    of    discharges    subject    to    binding  review. 
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Table  5  indicates  that  there  was  considerable  regional  varia- 
tion in  unit  costs.  HEW  Region  IX  had  the  highest  unit  costs  for 
total  hospital  review,  areawide  review,  and  concurrent  review  at 
$15.30,  $3.09,  and  $10.85  respectively.  Region  X  had  the  highest 
average  unit  MCE  cost  at  $2.30.  The  lowest  average  unit  costs  by 
region  were  as  follows:  total  hospital  review  -  $9.87  in  Region 
VII;  areawide  -  $1.12  in  Region  V;  concurrent  review  -  $6.68  in 
Region  VI;  and  MCE  -  $1.00  in  Region  IV. 

The  national  average  unit  cost  for  total  hospital  review 
for  conditional  PSROs  which  were  performing  review  at  the  end  of  FY 
19  77  was  $12.08.  The  range  went  from  a  low  of  $6.52  to  a  high  of 
$74.90.  The  latter  area  was  only  33  percent  implemented  by  the  end 
of  FY  1977,  however.  In  fact,  relatively  new  conditional  PSROs  most 
often  reported  very  high  unit  costs.  This  was  usually  due  to 
several  months  of  expenditures  prior  to  the  initiation  of  binding 
review.  Among  the  4  2  PSROs  which  were  more  than  90  percent  imple- 
mented by  the  end  of  FY  1977,  unit  costs  for  total  hospital  review 
were  most  commonly  reported  in  the  range  of  $9  to  $10,  with  about 
half  falling  between  $8  and  $12.  The  data  suggest  some  possible 
economies  of  scale. 

Table  6  indicates  the  ranges  of  review  costs.  The  median  of 
total  PSRO  review  unit  costs  was  $19.56.  The  corresponding  minimum 
and  maximum  figures  were  $9.70  and  $104.71  respectively.  Total  PSRO 
review  unit  costs  are  equal  to  the  sum  of  1)  management  and  support 
unit  costs  and  2)  total  hospital  review  unit  costs.  Note,  however, 
that  because  medians  are  used  in  the  table,  the  addition  of  manage- 
ment and  support  medians  and  total  hospital  review  medians  does  not 
necessarily  indicate  the  total  PSRO  review  median  unit  cost.  The 
median  unit  cost  of  management  and  support  activities  was  $5.07  and 
the  median  of  total  hospital  review  unit  cost  was  $12.31.  Median 
unit  costs  for  concurrent  review  varied  from  a  low  of  $3.25  to  a 
high  of  $18.09,  the  latter  being  in  a  PSRO  which  was  35  percent 
implemented  by  the  end  of  FY  1977.  The  median  unit  CR  cost  was 
$8.76  for  delegated  review  and  $10.94  for  nondelegated  review. 

There  was  more  variation  in  MCE  unit  costs  than  for  concurrent 
review.  One  reason  for  this  is  the  wide  range  in  implementation 
status  among  PSROs  and  the  fact  that  MCEs  are  usually  initiated  only 
after  concurrent  review  has  been  implemented.  A  second  reason  has 
to  do  with  the  determination  of  the  number  of  MCEs  required  in  any 
PSRO.  Because  this  numerical  requirement  is  based  on  the  size  of 
each  hospital  in  the  PSRO  area,  variation  in  size  of  hospitals 
within  PSROs  and  variations  in  the  number  of  hospitals  between  PSROs 
will  both  contribute  to  variation  in  unit  MCE  costs.  Unit  MCE  costs 
varied  from  a  low  of  $.08  to  a  high  of  $9.19  (this  particular  PSRO 
was  implemented  in  only  35  percent  of  its  hospitals  by  the  end  of  FY 
77).  The  median  unit  MCE  cost  was  $1.40.  Examining  delegated  and 
nondelegated  MCEs  separately  shows  median  unit  costs  of  $1.35  and 
$1.64,  respectively. 
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TABLE  5 


AVERAGE  HOSPITAL  REVIEW  UNIT  COST 
BY   HEW  REG  I  CN  AND  COMPONENT  OF  COST 
FY  1977 


Region 


Total 
Hospital 
Review/ 1 


Areawide 
Review 


Concurrent 
Review 


MCE 


i 

ii 
in 

IV 

V 

VI 

VII 

VIII 

IX 

X 

National/2 


$12.47 
14.40 
14.42 
11.30 

9.92 
10.03 

9.87 
12.55 
15.30 
11.26 
12.08 


$2.20 
2.84 
2.20 
1.74 
1.12 
1.73 
1.76 
1.13 
3.09 
1.69 
1.94 


$  9.16 
10.13 
10.66 
8.56 
7.32 
6.68 
6.94 
9.73 
10.85 
7.27 
8.78 


$1.11 
1.43 
1.56 
1.00 
1.48 
1.62 
1.17 
1.69 
1.36 
2.30 
1.36 


/1      Management  support  costs  are  excluded. 

/2       National  average  figures  were  computed  by  dividing  the  total 
costs    for   each   function   by   the  total  discharges   related  to 

those  specific  costs. 
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TABLE  6 

RANGE  IN  UNIT  COST  VARIATION  FOR  TOTAL  PSRO  REVIEW 
BY  COMPONENT  OF  COST  AND  DELEGATION  STATUS 
FY  1977 


Function 

Total  PSRO 

Revi  ew 
Management  and 

Support 
Total  Hospital 

Revi  ew 

Concurrent 
Review  (CR) 

MCE 

Areawide 
Revi  ew 


UNIT  COST  VARIATION/1 
Minimum  Median/2  Maximum 


$9.70 
1.11 

6.52 

3.25 
.08 

.08 


$19.56 
5.07 
12.31 

8.76 
1.40 

2.21 


$104.71 
81.44 
74.90 

18.09 
9. 19 

59.07 


UNIT  COST  VARIATION  BY  DELEGATION  STATUS 

Delegated  CR                   3.25                     8.76  18.09 

Nondelegated  CR             4.16                   10.94  81.13 

Delegated  MCE                   .20                     1.35  9.19 

Nondelegated  MCE              .08                     1.64  22.69 


/1  Because  no  single  PSRO  had  the  highest  unit  cost  for  all 
functions,  the  maximum  for  any  function  will  be  less  than 
the  sum  of  the  maxima  for  its  subcomponents.  (Similarly 
the  minimum  for  any  function  will  exceed  the  sum  of  the 
minima  of  its  subcomponents.)  In  addition,  it  is  inappro- 
priate to  add  medians  across  subcomponents. 

/2  Note  that  the  values  reported  are  median  values,  whereas  these 
in  the  previous  table  are  averages. 
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The  median  value  for  areawide  review  unit  cost  was  $2.21. 
Median  unit  costs  for  areawide  review  ranged  from  a  low  of  $.08  to  a 
high  of  $59.07.      (This  same  PSRO  had  the  high  MCE  cost  of  $9.19.) 

An  estimation  of  the  costs  of  the  program  if  it  had  been  fully 
implemented  during  1977  came  to  $235.7  million,  within  an  estimated 
range  of  $233.7  million  to  $239.9  million. 

1.4.3    Benefit-Cost  Analysis  of  Concurent  Review 

The  initial  objective  of  the  benefit-cost  analysis  was  to 
estimate  a  dollar  value  for  the  utilization  changes  attributable  to 
concurrent  review.  A  second  objective  was  to  test  the  sensitivity 
of  those  estimates  to  changes  in  the  assumptions  used  in  the  analy- 
sis. The  third  and  final  objective  was  to  compare  the  estimated 
value  of  program  benefits  to  the  costs  of  concurrent  review. 

This  analysis  was  designed  to  measure  benefits  in  a  very 
restricted  way.  The  only  benefits  considered  were  reductions  in 
Medicare  expenditures  for  health  care  services.  Other  potential 
benefits  to  Medicare  such  as  improved  quality  of  care  or  the  value 
to  beneficiaries  of  not  being  hospitalized  were  not  included  in  the 
analysis.  Savings  to  the  Medicaid  programs  attributable  to  PSRO 
could  not  be  calculated  due  to  unavailability  of  data.  Further,  PSRO 
impact  on  other  non-Federal  patient  populations  (i.e.,  patients  not 
subject  to  PSRO  review)  was  not  included.  It  is  not  clear  how  the 
latter  omission  might  change  the  results.  While  there  may  be 
positive  "spillovers"  to  non-Federal  patients  in  terms  of  quality 
improvements,  negative  spillovers  in  terms  of  increases  in  hospital 
unit  prices  to  non-Federal  patients  are  also  possible. /I 

Much  of  the  data  for  the  benefit-cost  analyses  came  from  other 
studies  in  the  1978  PSRO  evaluation,  including  the  100  percent 
Medicare  claims  utilization  study  and  the  analysis  and  synthesis  of 
program  costs.  In  order  to  incorporate  the  results  of  the  cost 
studies,  it  was  necessary  to  separate  out  the  costs  of  concurrent 
review  for  Medicare  only.  This  was  done  by  taking  a  proportion  of 
concurrent  review  costs  equal  to  Medicare's  proportion  of  all 
reviewed  Federal  patients.  Data  on  Medicare's  proportion  by  PSRO 
area  came  from  the  PMIS. 

A  further  adjustment  was  made  to  transform  fiscal  year  1977 
expenditures  to  calendar  year  1977  figures.  Quarterly  budget  data 
for  fourth  quarter  1976  through  fourth  quarter  1977  were  used  when 
available.  For  some  PSROs,  however,  the  adjustment  to  calendar  year 
19  77  was  based  on  the  increase  in  Medicare  beneficiaries  subject  to 
review.     This  adjustment  process  may  have  biased  upward  the  cost 


/1  When  a  hospital  experiences  decreases  in  its  days  of  care,  in 
the  short  term  its  per  diem  reimbursement  will  be  increased  in 
order  to  cover  fixed  costs.  Such  increases  in  per  diem  changes 
or  costs  affect  both  Federal  and  non-Federal  patients. 
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estimates  of  PSROs  which  experienced  substantial  increases  in 
implementation  in  calendar  year  1977.  This  is  because  the  unit 
review  cost  probably  declined  in  such  PSROs  due  to  a  spreading  of 
initial  costs  over  an  increasing  base  of  discharges  subject  to 
review.  Unfortunately,  it  was  not  possible  to  refine  the  adjustment 
process  to  account  for  these  economies  of  scale. 

The  other  major  source  of  data  for  the  benefit-cost  analyses 
was  a  file  of  audited  Medicare  cost  reports  for  some  5200  hospitals. 
These  data  were  incorporated  into  a  simulation  analysis  to  estimate 
the  relationship  between  changes  in  Medicare  hospital  utilization 
and  changes  in  Medicare  reimbursements. 

The  simulation  produced  several  significant  results.  The 
first,  a  measure  of  the  value  of  a  day  saved  in  each  PSRO,  was  used 
to  estimate  the  savings  to  Medicare  due  to  changes  in  hospital  days 
of  care  in  calendar  year  1977.  The  results  show  that  for  the  nation 
as  a  whole,  the  net  value  to  Medicare  of  a  day  saved  in  1977  was 
approximately  sixty  dollars.  This  estimate  represents  thirty-six 
percent  of  Medicare's  average  per  diem  reimbursement  to  short  stay 
hospitals  during  that  year. 

Second,  the  simulation  showed  that  the  estimates  of  the  value 
of  a  day  saved  were  very  sensitive  to  assumptions  about  changes  in 
ancillary  usage  and  hospital  expenditures.  Changes  in  those  assump- 
tions could  produce  estimates  of  the  value  of  a  day  saved  as  high  as 
sixty-two  percent  of  the  average  per  diem  or  as  low  as  nineteen 
percent  of  the  per  diem. 

Third,  the  estimates  were  also  found  to  be  very  sensitive  to 
assumptions  about  the  substitution  of  ambulatory  care  for  inpatient 
care.  For  example,  when  it  was  assumed  that  ambulatory  care  was 
rarely  substituted  for  hospital  care,  the  value  of  a  day  saved  was 
forty-five  percent  of  the  per  diem.  A  high  degree  of  substitution 
reduced  this  estimate  to  twenty-seven  percent.  Yet  the  results  were 
found  to  be  relatively  insensitive  to  assumptions  about  the  substi- 
tution of  long-term  care  for  short-stay  care. 

Fourth,  the  estimates  were  found  to  be  moderately  sensitive 
to  assumptions  about  the  relative  costs  of  the  hospitals  where 
utilization  reductions  might  be  obtained.  The  results  were  rela- 
tively insensitive  to  assumptions  based  on  the  hospitals'  occupancy 
rates  or  average  lengths  of  stay. 

These  simulation  results  were  used  to  estimate  the  value  of 
reductions  in  hospital  utilization  in  each  PSRO  area.  These  esti- 
mates were  then  compared  to  the  cost  estimates  for  Medicare  for 
total    concurrent    review.        The    following    results    were  obtained: 
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•  If  all  PSROs  had  exactly  the  same  impact  on  Medicare  days 
of  care  per  thousand,  then  a  reduction  of  1 .66  percent 
would  be  required  for  benefits  to  equal  costs.  In  the 
1977  Evaluation  this  breakeven  point  was  estimated  at 
2.05  percent.  Because  PSRO  costs  have  been  increasing 
less  rapidly  than  hospital  costs,  PSROs  are  becoming 
potentially  more  efficient. 

•  If  the  average  unit  costs  of  concurrent  review  could  be 
reduced  to  achieve  the  FY  1979  cost  target  (of  $8.70  for 
total  hospital  review),  the  breakeven  point  could  be 
attained  with  a  reduction  in  days  of  care  per  thousand  of 
only  1.34  percent. 

•  An  average  relative  reduction  in  days  of  care  of  1.5 
percent  was  observed  in  the  96  conditional  PSROs  active 
as  of  April  1977.  The  value  of  this  reduction  nationally 
was  $50.5  million.  Estimated  total  costs  for  concurrent 
review  in  those  areas  came  to  $45  million.  Hence,  the 
aggregate  benefit-cost  ratio  was  1.10. 

•  Using  the  results  of  the  disaggregated  utilization 
analysis,  30  PSROs  had  average  benefit-cost  ratios  in 
excess  of  one.  The  highest  benefit-cost  ratio  observed 
was  approximately  12  to  1 . 

•  If  all  PSROs  could  have  been  brought  up  to  at  least  the 
average  reduction,  then  fifty-two  of  the  ninety-six  PSROs 
would  have  had  benefit-cost  ratios  in  excess  of  1.  The 
aggregate  benefit  cost  ratio  in  such  an  event  would  have 
been  1.51. 


1.4.4    Medical  Care  Evaluation  Studies 

Medical  Care  Evaluation  Studies  (MCEs)  are  the  PSRO  program's 
principal  means  of  monitoring  and  improving  the  quality  of  care 
provided  in  hospitals  which  treat  Federal  beneficiaries.  The  basic 
mechanism  is  an  adaptation  of  the  medical  audit,  a  method  of  peer 
review  which  has  gained  increasing  acceptance  in  recent  decades. 
Generally,  medical  audits  involve  the  specification  of  a  topic, 
establishment  of  criteria  of  good  care,  a  sampling  of  records, 
comparison  of  criteria  with  actual  performance,  an  analysis  of 
findings,  and  recommendations  for  action  to  correct  deficiencies 
which  have  been  detected.  There  are  four  significant  differences 
instituted  by  the  PSRO  program  as  compared  with  traditional  audits: 

(1)  PSROs  require  reaudits  to  confirm  the  effectiveness  of 
recommended  actions;  (2)  PSROs  are  responsible  for  the  number 
and   quality    of    MCEs    done,    either   by   conducting   the  studies 
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themselves  or  by  delegating  the  task  to  qualified  hospitals; 
(3)  PSROs  facilitate  areawide  (multiple-site)  MCEs;  and  (4) 
HSQB  provides  program  monitoring  and  technical  assistance  to 
ensure  more  uniform  MCE  study  activities. 

The  evaluation  of  the  MCE  component  of  the  PSRO  program  in- 
cludes two  approaches.  The  first  is  a  descriptive  study  of  the 
nationwide  implementation  of  MCEs  through  analysis  of  reports 
gathered  by  the  PMIS.  Numbers  of  initial  audits  and  reaudits 
completed  are  reported  over  time,  by  PSRO,  and  by  topic  categories. 
The  second  study,  more  experimental,  tests  a  methodology  which 
attempts  to  measure  the  impact  of  MCEs  by  comparing  the  degree  of 
deviation  from  criteria  of  good  care  before  and  after  corrective 
actions  are  taken. 

1.4.4.1     Quantity  and  Distribution  of  MCE  Activity 

The  number  of  initial  MCE  audits  reported  through  the  PMIS  has 
increased  each  quarter  (with  one  unexplained  exception)  during  the 
period  for  which  data  have  been  collected,  the  second  quarter  of 
1975  through  the  second  quarter  of  1978.  This  trend  is  explained 
both  by  an  increasing  number  of  PSROs  reporting  MCEs  and  by  an 
increasing  average  number  of  MCEs  reported  per  PSRO.  PSRO  produc- 
tivity is  uneven,  however;  of  109  PSROs  reporting  between  1975  and 
1977,    the    top    10    PSROs    accounted   for    35   percent    of    all  audits. 

Reaudit  activity  is  less  impressive:  only  414  reaudit  reports 
were  entered  in  the  PMIS  computer  file  for  1975-1977.  Hand  counts 
of  reports  not  yet  entered  revealed  an  additional  784  reaudits. 
Nevertheless,  the  total  is  still  small  considering  the  fact  that 
fully  70  percent  of  initial  audit  reports  indicate  findings  which 
require  reaudits. /1 

An  examination  of  the  topics  of  MCE  initial  audits  for  1975- 
1977  found  that  48.6  percent  of  audits  were  done  on  medical  topics 
and  42.8  percent  on  surgical  topics,  while  the  remaining  8.6  percent 
focused  on  administrative  and  miscellaneous  procedures.  When  the 
audit  topics  were  classified  into  568  categories,  the  20  most 
frequently  audited  topic  categories  for  1975  through  1977  accounted 
for  43.6  percent  of  all  studies  done.  Comparing  this  percentage 
with  the  1977  evaluation  finding  that  56  percent  of  the  MCE  studies 
conducted  in  1975  and  1976  were  of  the  20  most  frequent  topic 
categories  suggests  that  there  has  been  a  diversification  in  topic 
selection  over  time.  Myocardial  infarction,  pneumonia,  cholecystec- 
tomy,   and   appendectomy   are    consistently   among  the   five   most  fre- 


/1  The  proportion  of  indicated  reaudits  which  have  been  completed 
is  not  readily  computed,  since  reaudits  may  be  done  up  to  one 
year  after  the  initial  audit  has  been  completed.  Looking  only 
at  audits  completed  through  the  second  quarter  of  1977,  18 
percent  of  the  expected  number  of  reaudits  had  been  reported 
by  the  second  quarter  of  1978. 
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quently  audited  topic  categories,  but  audits  of  medicinal  agents 
show  a  strong  increasing  trend. 

1*4.4.2    Pilot  Variation  Rate  Analysis 

Systematic  assessment  of  the  impact  of  the  quality  assurance 
program  is  yet  in  its  infancy.  The  results  reported  here  reflect 
the  efforts  of  a  pilot  study  to  measure  MCE  impact  by  analyzing 
changes  in  variation  rates  (VRs).  VRs  are  rates  of  deviation  from 
standards  of  care,  compared  for  each  criterion  of  care  at  both  the 
Initial  audit  and  the  reaudit/1 .  The  difference  between  the  initial 
audit  VR  and  the  reaudit  VR  suggests  changes  in  the  level  of  care. 
If  VRs  fall,  deviations  from  standards  are  reduced  and  care  is 
thought  to  be  improved. 

The  purpose  of  the  pilot  study  is  primarily  methodological  in 
that  it  assists  in  the  design  of  a  more  comprehensive  study  of  VR 
changes  to  be  completed  in  1979.  The  study  is  a  secondary  analysis; 
data  are  limited  to  those  routinely  available  through  the  PMIS  and 
background  documentation  (audit  forms,  committee  minutes,  etc.) 
solicited  from  the  participating  PSROs.  The  low  frequency  of 
reaudits  necessarily  restricted  the  number  of  possible  pre-post 
comparisons  and  confined  the  sample  of  PSROs  to  those  which  have 
produced  the  most  reaudits  in  the  chosen  topic  categories.  For 
reasons  of  the  small  and  unrepresentative  sample,  limitations  of 
design,  and  lack  of  control  over  data  quality,  the  temptation  to 
draw  substantive  conclusions  from  the  study  must  be  carefully 
restrained.  At  most,  the  findings  regarding  VRs  among  the  pilot 
sample  may  give  some  indication  of  the  patterns  to  be  found  in 
the  more  conclusive  studies  which  are  to  follow. 

The  pilot  study  data  base  includes  changes  in  VRs  (audit  to 
reaudit)  for  136  MCEs  done  in  8  PSROs  in  6  topic  categories  re- 
flecting 52  common  criteria.  In  every  topic  category,  changes  in 
VRs  were  calculated  for  each  criterion  of  care  which  was  included  in 
the  sampled  audits  and  reaudits  with  sufficient  frequency  to  yield 
reliable  rates.  Topic  categories  studied  included  those  most 
frequently  reaudited  in  the  participating  PSROs:  appendectomy, 
cholecystectomy,  diabetes,  hysterectomy,  myocardial  infarction,  and 
pneumonia.  Variation  rates  pooled  across  study  sites  were  computed 
for  the  52  criteria  of  care  which  were  audited  and  reaudited  with 
sufficient  frequency  within  these  categories.  Comparisons  were  made 
only  for  sites  that  used  a  criterion  both  at  audit  and  at  reaudit. 
The  decision  was  made  to  examine  rates  of  variation  from  standards 


/1  Since  an  MCE  generally  involves  the  comparison  of  pre-estab- 
lished explicit  criteria  of  care  with  actual  performance  as 
reflected  in  the  medical  records  of  patients  with  a  given 
condition,  procedure,  or  diagnosis,  each  topic  category  has  a 
number  of  criteria  associated  with  it. 
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on  these  criteria  regardless  of  whether  these  variations  were  held 
by  the  auditing  physicians  to  be  justified,  since  the  process  of 
justification  is  usually  undocumented  and  would  introduce  another 
source  of  uncontrolled  variability. 

Since  a  quality  assurance  program  cannot  be  expected  to 
affect  all  dimensions  of  care  equally,  criteria  were  separated 
into  three  classes  which  vary  in  the  degree  to  which  they  are 
amenable  to  control.  The  first  class,  medical  care  process,  was 
thought  to  be  most  clearly  under  provider  control.  Outcome  criteria 
were  divided  into  those  preventable  with  proper  care  and  those 
judged  to  be  non-preventable. 

Study  results  are  expressed  in  terms  of  the  aggregate  change  in 
VR  from  audit  to  reaudit  for  each  criterion.  Decreases  in  VRs  imply 
improvement  in  measured  performance,  while  increases  imply  poorer 
performance  at  reaudit  than  at  audit.  The  area  of  "no  change" 
includes  either  zero  change  in  VRs  or  changes  that  were  less  than 
the  aggregate  rounding  error.  Table  7  presents  the  changes  in  VRs 
observed  for  the  three  groups  of  criteria. 


TABLE  7 

NUMBER  OF  MCE  STUDIES  EXHIBITING  CHANGES  IN  VARIATION  RATE  BETWEEN 
AUDIT  AND  REAUDIT,   BY  CRITERIA  GROUP 


Number 

of  Studies 

with  Results  Shown  Below 

Type  of 
Criterion 

Increased 
VR 

No 
Change 

Decreased 

VR 

.01- 

.05  t 

.00+ 
Error* 

.01- 
.05 

.06- 
.10 

.11- 
.15 

.16- 

.20 

.21- 
.25 

.26- 
.30 

31- 

35 

.36- 
.40 

.41- 
.45 

Process 

2 

4 

3 

7 

5 

4 

3 

2 

0 

0 

1 

Preventable 
Outcome 

0 

7 

6 

1 

0 

0 

0 

0 

0 

1 

0 

Non-Preventat 
Outcome 

le 

0 

4 

2 

0 

0 

0 

0 

0 

0 

0 

0 

TOTAL 

2 

15 

11 

8 

5 

4 

3 

2 

0 

1 

1 

•Included  as  "no  change"  are  those  increases  or  decreases  which  do  not  exceed  an  estimated  rounding 
error. 
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Process  criteria,  as  expected,  show  the  greatest  improvement. 
While  two  criteria  show  increased  VRs  from  audit  to  reaudit  and  four 
VRs  essentially  remained  the  same,  the  remaining  (25)  process 
criteria  improved,  one  of  them  by  as  much  as  44  percent.  Outcomes 
judged  amenable  to  provider  control  also  changed  favorably.  Non- 
preventable  outcomes  exhibit  no  trend. 

Interpretation  of  the  VR  findings  is  difficult.  If  the  VR 
findings  are  valid,  they  indicate  that  for  a  small  and  unrepresen- 
tative sample  of  MCEs  examined  at  two  points  in  time  without  a 
control  group,  care  came  increasingly  within  quality  standards  for 
the  criteria  examined.  The  validity  of  the  underlying  data  is, 
however,  a  serious  question.  MCE  study  reports  were  not  designed  to 
produce  data  for  evaluative  purposes  and  these  data  are,  therefore, 
subject  to  unknown  inconsistencies  and  biases. 

To  test  the  validity  of  VR  calculations,  reviewers  were  asked 
to  examine  a  subset  of  studies  in  two  topic  categories  to  ascertain 
whether  changes  in  criteria,  specifications,  or  data  collection 
instructions  between  audit  and  reaudit  could  account  for  changes 
in  variation  rates.  In  87  percent  of  the  79  instances  in  which 
criteria  were  applied  both  at  audit  and  at  reaudit,  the  reviewers 
found  that  criteria,  specifications,  and  data  collection  instruc- 
tions had  not  changed  between  audit  and  reaudit.  When  one  or  more 
of  these  factors  had  changed  between  audit  and  reaudit,  the  re- 
viewers judged  that  the  change  would  decrease  the  reaudit  VR  in 
three  instances,  not  affect  it  in  four  instances,  and  create  un- 
certain changes  in  three.  Thus,  this  crude  test  of  audit/reaudit 
reliability  does  not  support  the  hypothesis  that  the  change  in 
observed  VRs  can  be  accounted  for  by  systematic  bias  due  to  loosen- 
ing of  criteria  at  reaudit. 

On  the  other  hand,  when  reviewers  were  asked  whether  they 
judged  the  observed  changes  in  VRs  for  the  same  sample  of  criteria 
to  be  valid,  they  supported  the  validity  of  the  findings  only  63 
percent  of  the  time.  The  main  reasons  given  for  invalidity  were  the 
changes  in  criteria,  exceptions,  or  instructions  as  noted  in  the 
reliability  study  and  the  absence  of  data  gathering  instructions  at 
both  audit  and  reaudit  for  criteria  which  required  interpretation  in 
their  application. 

If  the  data  from  this  pilot  study  are  to  be  believed,  then  MCEs 
will  most  likely  show  impact  on  the  process,  if  not  the  outcomes  of 
care  in  more  thorough  studies  of  the  MCE  process.  In  order  to  be 
more  credible,  however,  such  studies  must  either  have  access  to  more 
valid  and  reliable  data  than  are  routinely  recorded  on  medical  audit 
forms,  or  engage  in  extensive  re-abstracting  studies  to  try  and 
validate  the  information  found  there.  The  MCE  evaluation  study  to 
be  completed  in  1979  has  already  incorporated  these  requirements  in 
its  study  design. 
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The  main  conclusion  to  be  drawn  from  the  pilot  study  is  that 
the  data  generated  by  the  MCE  process  are  not  adequate  to  link 
definitively  the  MCE  process  to  changes  in  the  quality  of  care 
delivered*  The  results  tentatively  suggest  that  MCEs  are  improving 
the  quality  of  care,  but  this  is  by  no  means  certain.  If  quality 
assurance  programs  are  to  meet  standards  of  accountability  that  are 
required  of  utilization  control  programs,  better  data  must  be  made 
available* 

1.4.5    Program  Management  Experience 

1.4.5.1    Management  Assessments 

Between  November  1976  and  October  1978,  HSQB  conducted  twenty- 
six  intensive,  on-site  assessments  of  PSRO  performance  which  focused 
on  PSRO  management.  These  site  assessments  generally  lasted  three 
days.  The  assessment  teams  consisted  of  five  persons,  including 
both  Federal  and  non-Federal  personnel,  as  well  as  representatives 
from  other  PSROs.  The  1978  evaluation  undertook  three  different 
analyses  of  the  data  contained  in  the  written  site  assessments. 
While  the  limited  number  of  these  reports  precludes  drawing  con- 
clusions about  the  program  as  a  whole,  the  results  indicate  that  the 
site  assessment  process  can  identify,  on  a  reasonably  systematic 
basis,  factors  which  affect  PSRO  management  performance. 

The  ratings  of  PSRO  performance  on  the  individual  items  which 
are  assessed  tend  to  be  more  or  less  normally  distributed,  a  factor 
which  affirms  the  utility  of  the  assessment  reports  for  analysis. 
The  items  which  are  not  normally  distributed  and  tend  to  be  rated 
low  are  those  aspects  of  the  PSRO  program  which  were  not  emphasized 
early  in  the  program  such  as  MCEs,  profiling,  and  focusing  of 
review. 

Overall,  the  twenty-six  PSROs  received  high  ratings  in  the 
area  of  competence  of  executive  leadership,  data  system  capability, 
implementation  of  hospital  review  as  projected,  keeping  hospitals 
informed  regarding  their  performance,  and  in  perceived  PSRO  per- 
formance held  by  other  health  care  organizations. 

On  the  other  hand,  PSROs  tended  to  receive  low  scores  on 
withdrawal  of  delegation  from  hospitals,  all  items  dealing  with  MCE 
studies,  ancillary  services  review,  review  of  physician  services, 
and  items  related  to  denials  and  sanctions.  Specifically,  the 
analyses  of  the  assessments  reports  found  the  following: 

•  Two-thirds  of  the  PSROs  assessed  had  hospital  review 
programs  prior  to  becoming  a  PSRO.  This  prior  experience 
appears  to  have  been  beneficial  in  implementation  of  PSRO 
hospital  review. 
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•  Assessors  were  complimentary  about  the  competence, 
dedication  and  knowledge  of  PSRO  executive  boards. 
About  90  percent  of  board  membership  is  comprised  of 
physicians. 

•  Assessors  viewed  PSRO  staff  as  competent,  but  noted 
a  consistent  lack  of  specific  training  for  many  staff 
positions,  such  as  physician  advisors,  nurse  coor- 
dinators, and  executive  board  members. 

•  Physician  inputs  from  the  community-at-large  were  judged 
by  the  site  teams  as  minimal,  though  no  outright  resis- 
tance to  PSRO  activity  was  indicated. 

•  Assessors  found  that,  at  the  time  of  assessment,  MCEs 
were  being  performed  primarily  at  the  hospital  level 
rather  than  on  an  areawide  basis.  Deficiencies  were 
noted  in  all  hospital  MCE  activities. 

•  In  evaluating  the  hospital  review  programs,  the  site 
assessors  noted  problems  with  physician  advisors,  in- 
cluding inappropriate  selection  procedures,  poor  training 
methods,  and  poor  communication  with  review  coordinators. 

•  Site  assessment  teams  noted  that  the  establishment  of 
areawide  norms  and  criteria  was  a  general  problem, 
although  a  significant  number  of  PSROs  were  attempting  to 
systematize  both  norms  and  criteria. 

•  Focused  review  and  profile  analysis  were  found  to  be 
operating  more  frequently  in  older  PSROs. 

•  The  most  salient  and  frequently  cited  problem  in  the 
assessments  was  the  lack  of  formal  PSRO  monitoring  of  the 
costs  and  the  effectiveness  of  hospital  review. 

•  The  data,  while  limited,  indicate  a  relationship  between 
the  age  of  the  PSRO  and  its  performance  on  the  various 
components  studied. 

In  summary,  the  analyses  show  that  there  are  some  major  dif- 
ferences in  the  way  in  which  PSROs  are  managed  and  there  are  a 
series  of  problem  areas  which  are  common  to  all  PSROs  whether  they 
are  well  managed  or  not. 

1.4.5.2    Project  Officer  Ratings 

Assessments  of  individual  PSRO  performance  in  nine  categories 
of  managerial  and  operational  activities  were  made  by  DHEW  regional 
office  PSRO  project  officers  for   107  of  the   108  PSROs  which  had 
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conducted  any  hospital  review  in  19  77.  The  data  are  analyzed  in 
light  of  two  objectives:  (1)  to  determine  the  current  (as  of 
October,  1978)  status  of  PSRO  management  performance  and  (2)  to 
determine  factors  which  may  help  interpret  the  findings  of  the 
preceding  utilization,  cost,  and  MCE  studies.  Since  the  ratings  are 
for  19  78,  they  cannot  be  directly  correlated  with  the  1977  utiliza- 
tion analyses  in  this  report.  They  do,  however,  provide  the  reader 
with  information  about  the  organizational  aspects  of  the  PSRO  and 
provide  a  context  for  understanding  the  analyses  described  above. 

1.4.6     1978  Program  Status  and  Directions 

The  PSRO  program  has  progressed  considerably  since  the  1977 
evaluation.  There  have  been  organizational  changes,  further  imple- 
mentation of  review,  and  significant  efforts  to  improve  program 
performance.  Organizationally,  HSQB  was  transferred  from  the  Public 
Health  Service  (PHS)  to  the  Health  Care  Financing  Administration 
(HCFA)  in  March  19  77.  To  consolidate  all  PSRO  activities  in  one 
agency,  the  staffing  and  support  functions  for  the  National  Pro- 
fessional Standards  Review  Council  (NPSRC)  were  also  transferred 
from  PHS  to  HCFA  on  June  2,  1978. 

Several  PSRO  area  redesignations  have  been  made,  the  most 
significant  being  the  consolidation  of  the  original  nine  areas  in 
Texas  to  a  single  statewide  PSRO  area,  leaving  a  total  of  195  areas 
nationwide.  Other  area  refinements  have  been  made  including  reduc- 
ing Minnesota  from  three  areas  to  two  and  increasing  Ohio  from 
twelve  areas  to  thirteen.  The  redesignations  in  Texas  and  Minnesota 
eliminate  the  requirement  for  Statewide  Professional  Standards 
Review  Councils  (SPSRCs)  in  those  states.  Of  the  sixteen  states 
required  to  establish  SPSRCs,  eight  have  been  appointed,  seven 
others  are  pending,  and  Louisiana  has  not  yet  begun  the  appointment 
process. 

Fiscal  year  19  78  funding  of  the  PSRO  program  (including  both 
Congressional  appropriations  and  Medicare  Trust  Fund  payments)  was 
estimated  at  $147  million  with  an  estimated  budget  of  $174  million 
for  fiscal  year  1979.  Organizational  funding  has  progressed  signif- 
icantly with  178  conditional  PSROs,  15  planning  PSROs,  and  2  un- 
funded areas  as  of  October  1,  1978. 

In  an  effort  to  improve  program  performance,  HSQB  has  estab- 
lished program  initiatives  concerning  three  areas:  (1)  implementa- 
tion objectives,  (2)  general  management  objectives,  and  (3)  improve- 
ment in  the  review  process. 

The  program  has  implementation  objectives  to  place  conditional 
PSROs  in  all  195  areas  and  assure  that  4,900  hospitals  (82%)  are 
under  review  by  March  31,  1979.  It  appears  that  the  objective  of 
funding  195  conditional  PSROs  will  be  met,  but  given  that  hospital 
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implementation  is  at  70  percent  in  conditional  areas  as  of  September 
30,  1978,  increased  activity  will  be  needed  to  meet  the  goal  of  82 
percent. 

The  general  management  initiatives  consist  of  ( 1 )  the  estab- 
lishment of  performance  objectives,  (2)  improved  financial  manage- 
ment, and  (3)  improved  relationships  with  Health  Systems  Agencies 
and  Medicaid  State  Agencies.  The  performance  objective-setting 
process  basically  identifies  local  problem  areas  within  the  context 
of  national  concerns,  establishes  measurable  objectives  against 
which  to  gauge  improvement,  and  works  toward  those  objectives. 
Despite  observations  that  initial  objectives  have  not  been  specific 
enough,  the  objective-setting  process  is  expected  to  improve.  The 
major  cost  reduction  effort  focuses  on  unit  costs  for  hospital 
review  with  a  goal  of  $8.70  cost  per  discharge  by  September  30, 
1979.  Review  improvements  are  being  made  in  the  areas  of  focused 
review,  profile  analysis,  MCE  studies,  and  model  criteria  set 
development  for  surgical  review.  Specific  examples  of  PSRO  review 
activities  demonstrate  that  PSROs  can  effectively  focus  on  problem 
areas  and  produce  effective  correction  action. 

A  program  of  PSRO  demonstration  projects  is  being  conducted  to 
develop  review  methodologies  in  several  areas  and  to  help  shape 
Departmental  policy.  These  demonstrations  include  fifteen  long  term 
care  review  projects,  five  ambulatory  care  review  projects,  ten 
ancillary  services  review  projects,  twelve  focused  review  projects, 
and  five  surgical  evaluation  projects. 

While  the  program  initiatives  and  demonstration  projects 
cannot  be  assessed  in  this  evaluation,  they  do  offer  directions  for 
the  future  development  of  the  program.  Future  evaluations  may 
attempt  to  measure  the  progress  of  the  program  in  meeting  its  estab- 
lished objectives  and  the  implications  of  those  objectives  for 
overall  program  performance. 

1.4.7    Directions  for  Future  Evaluation 

The  1979  evaluation  will  continue  to  refine  and  develop  pre- 
vious work.     New  areas  to  be  investigated  will  include: 

•  The  effects  of  1978  program  initiatives; 

•  The  relationship  between  the  PSRO  program  and  other  cost 
containment  mechanisms; 

The  effect  of  focused  review  on  program  costs; 
Hospital  expenditure  patterns; 

•  Long-term    care    and    ambulatory    care    review  mechanisms; 
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•  MCE  variation  rates; 

•  Profile  analysis; 

•  Unnecessary  care; 

•  PSRO   alternatives    -   Oklahoma,    New   York,    and  second 
opinion  surgery  demonstrations. 

The  1979  PSRO  evaluation  will  continue  to  emphasize  the  bene- 
fits and  costs  of  utilization  control.  This  will  be  more  difficult 
as  inactive  PSRO  comparison  areas  become  active.  New  longitudinal 
analysis  will  be  required  to  work  with  data  sets  which  do  not 
contain  inactive  PSRO  areas.  An  exploratory  benefit-cost  analysis 
of  MCE  activity  will  be  presented  in  1979.  This  effort  will  attempt 
to  broaden  the  nature  of  program  benefits  incorporated  into  the 
evaluation.  The  exploration  of  unnecessary  care,  if  successful, 
will  enable  the  PSRO  program's  impact  and  potential  to  be  placed 
into  better  perspective  in  that  estimated  utilization  impacts  can  be 
compared  to  the  proportion  of  unnecessary  care  which  could  theo- 
retically be  eliminated  by  the  PSRO  program. 

1.5    Report  Organization 

The  remainder  of  the  volume  provides  the  technical  discussions 
which  support  the  evaluation  findings  summarized  above.  Section 
2  presents  the  utilization  studies,  where  major  emphasis  is  placed 
on  the  Medicare  100%  claims  file  analysis.  Section  3  describes 
the  program  costs;  these  are  then  combined  with  the  utilization 
impact  results  to  provide  a  benefit-cost  analysis  of  concurrent 
review,  presented  in  Section  4.  Sections  5  and  6  continue  with  the 
quality  assurance  studies  and  the  program  management  assessment.  A 
programmatic  perspective  is  then  provided  in  Section  7.  The  volume 
is  concluded  with  a  brief  description  of  future  evaluation  issues. 
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2.     UTILIZATION  IMPACT  STUDIES 


2.1  Overview 

2.1.1     Study  Objectives 

The  primary  objective  of  the  studies  reported  in  this  section 
is  to  determine  whether  or  not  PSRO  activities  have  resulted  in  a 
decrease  in  hospital  use.  A  key  assumption  underlying  the  objective 
is  that  some  proportion  of  hospital  use  is  unnecessary  and  that 
reductions  in  hospital  use  do  not  necessarily  imply  reductions  in 
the  quality  of  care  received  by  patients  in  the  hospital  setting. 
Although  the  methodological  issues  related  to  the  measurement  of 
inappropriate/unnecessary  care  have  by  no  means  been  definitively 
resolved,  several  authors  have  independently  performed  studies 
to  define  and  measure  inappropriate  utilization.  / 1-7  Their 
methods  have  varied  and  the  derived  values  range  from  9  to  20 
percent  of  total  days.  The  similarity  in  the  magnitude  of  these 
independently  derived  estimates  lends  credence  to  the  contention 
that  unnecessary  care  at  the  acute  care  level  is  being  rendered 
in  American  hospitals.     The  following  utilization  studies  are 


/1  Browning,  F.E.  and  Coump,  L.  Report  to  the  Patient  Care  Planning 
Care  Council  on  a  Bed  Utilization  Study  (Rochester,  N.Y.,  Council 
of  Social  Agencies  1969).  Mimeographed. 

/2  Gertman,  P.M.  and  Bucher,  P.M.  "Inappropriate  Hospital-Bed 
Days,  and  their  Relationship  to  Length  of  Stay  Parameters", 
presented  at  the  Medical  Care  Section,  99th  meeting  of  the 
American  Public  Health  Association,  Minneapolis,  Minnesota 
(October  1977). 

/3  Quenido,  A.  "Indirect  Evaluation  of  a  Randomized  Sample  of  the 
General  Practitioner's  Work  (1954)".  The  Efficiency  of  Medical 
Care,  Leiden,  H.H.,  Stenfen  Knoese,  N.V.  1963. 

/4  Restuccia,  J.D.  and  Holloway,  D.C. ,  "Barriers  to  Appropriate 
Utilization    of    an   Acute   Facility".      Medical  Care,    July  1976. 

/5  Rosser,  A.  "The  Reliability  and  Application  of  Clinical  Judg- 
ment in  Evaluating  the  Use  of  Hospital  Beds".  Medical  Care, 
January  1976. 

/6  Zimmer,  J.G.  and  Groomes,  E.W.  "An  Observer  Reliability  Study 
of  Physicians'  and  Nurses'  Decisions  in  Utilization  Review  of 
Chronic   Care   Facilities".      Medical  Care,   January -February  1969. 

/7  Zimmer,  J.G.,  "Length  of  Stay  and  Hospital  Bed  Misutilization" 
Medical  Care,  May  1974. 
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based  on  the  assumption  that  that  contention  is  correct  and  that 
reductions  in  hospital  days  of  care  are  generally  appropriate  as  a 
cost-saving  mechanism. 

2.1.2    Studies  and  Relationships 

This  1978  utilization  evaluation  consists  of  one  study  of  PSRO 
effectiveness  using  100  percent  Medicare  claims  data  and  three 
methodological  studies  designed  to  explore  the  possibility  of  using 
alternate  data  sets  and  methodologies  for  future  PSRO  evaluations. 

These  studies  are  defined  below: 

EVALUATION  STUDY 

Use  Study  1-100  Percent  Medicare  Claims  File  Study  (Section  2.2) 
METHODOLOGICAL  STUDIES 

Use  Study  2-20  Percent  Medicare  Discharge  File  Study  (Section  2.3) 

Use  Study  3  -  MEDPAR-PHDDS  Study  (Section  2.4) 

Use  Study  4  -  PHDDS  Staging  Study  (Section  2.5) 

Of  these  studies,  the  100  percent  Medicare  claims  file  analysis 
(Use  Study  1)  uses  the  most  recent  data  available  and  covers  the 
largest  number  of  active  PSROs.  All  Medicare  discharges  from 
calendar  years  1974  through  1977/1  were  aggregated  to  the  PSRO 
level  and  all  PSRO  areas  in  the  country  are  included  in  the  analy- 
sis./^ Therefore  this  study  has  the  greatest  implications  for 
measuring  the  impact  the  PSRO  program  has  had  on  Medicare  inpatient 
use  rates. 

The  basic  comparison  in  this  study  is  between  active  and 
inactive  PSRO  areas.  In  addition,  PSRO  effectiveness  as  a  function 
of  age  (length  of  time  that  the  PSRO  has  been  operating)  and  inten- 
sity (percent  of  all  Medicare  discharges  under  review)  is  tested. 
PSROs  which  had  at  least  one  hospital  under  binding  review  as  of 
April  1,  1977  were  considered  to  be  active  (n=96) ;  all  others  were 
considered  to  be  inactive  (n=93). 


/1  Medicare  discharges  for  the  second  half  of  1977  were  estimated 
from  admissions  for  that  year  due  to  lags  in  the  reporting  of 
discharges.  This  procedure  is  quite  reliable,  with  a  correlation 
between  admissions  and  discharges  of  .99. 

/2  The  actual  number  of  PSROs  in  the  analysis  (n=189)  differs 
slightly  from  the  203  designated  in  1977.  This  is  explained  in 
greater  detail  in  the  footnote  on  page  4. 
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Although  the  discharge  rate  (number  of  discharges  per  1000 
Medicare  enrollees)  and  average  length  of  stay  were  also  analyzed, 
the  emphasis  in  this  study  is  on  the  days  of  care  rate.  The  dis- 
charge rate  and  average  length  of  stay  can  fluctuate  independently 
(and  quite  possibly  in  opposite  directions)  so  that  neither  one 
alone  is  an  accurate  measure  of  whether  total  Medicare  inpatient  use 
is  increasing  or  decreasing.  Consequently,  the  days  of  care  rate, 
which  is  the  product  of  the  other  two  variables,  is  the  most  rele- 
vant measure  of  the  level  of  inpatient  care  provided.  It  is  also 
most  closely  related  to  expenditures  for  care. 

Multiple  regression  estimation  methods  were  used  in  Use  Study  1 
to  measure  the  effect  of  PSRO  review  on  the  total  days  of  care  per 
thousand  (DOC),  discharges  per  thousand  (DISC)  /1_  and  average  length 
of  stay  (ALOS)  after  controlling  for  certain  available  adjustment 
variables  /2  thought  to  influence  utilization.  The  adjustment 
variables  used  in  the  analysis  include  population  density,  poverty 
level,  physicians/1000  population,  short  stay  beds/1000  population, 
nursing  home  beds/1000  population,  short  stay  hospital  occupancy 
rate,  percent  of  population  who  are  aged,  and  percent  of  total 
admissions  which  are  Medicare.  A  variety  of  regression  specifica- 
tions was  tested  and  a  primary  model  was  selected. 

Results  based  on  the  primary  model  are  also  provided  in  dis- 
aggregated form  in  which  individual  PSRO  area  impacts  were  esti- 
mated. These  analyses  permitted  a  ranking  of  individual  PSRO  area 
impact  and  were  necessary  for  the  benefit-cost  study  (Section  4). 
Supplementary  analyses  were  also  conducted  concerning  regional 
variation,  the  relationship  between  PSRO  impact  and  prospective 
reimbursement,  the  change  in  PSRO  impact  over  time  and  the  impact  of 
days  certified  at  the  skilled  nursing  facility  level  of  care  but 
paid  for  in  acute  care  settings. 

Certain  characteristics  of  the  1977  Medicare  Utilization  Study 
led  to  the  development  of  Use  Study  2,  the  20  Percent  Medicare 
Discharge  File  Study  which  uses  1971  to  1973  and  1976  calendar 
year  data.  The  1977  Medicare  Study  examined  total  utilization 
rates,  but  did  not  take  into  consideration  changes  in  rates  by 
diagnostic  category  or  surgical  procedure.  However,  if  inappro- 
priate hospitalization  is  not  constant  across  all  types  of  illness 
categories    or    surgical    procedures,    decreased   utilization    in  one 


/1  All  subsequent  references  to  days  of  care  and  discharges  will 
be  in  terms  of  rates  per  thousand  Medicare  beneficiaries  unless 
otherwise  indicated. 

/2  Adjustor  or  control  variables  are  those  variables  which  might 
conceivably  affect  utilization,  but  are  essentially  beyond 
the  control  of  the  PSRO  program. 
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diagnostic  or  procedural  category  may  be  obscured  in  the  overall 
utilization  statistics  due  to  increased  utilization  in  another 
category.  For  instance,  a  decrease  in  days  of  care  for  chronic 
ischemic  heart  disease  may  be  balanced  by  an  increase  in  days  of 
care  for  cataract  surgery.  Certain  rates  can  even  be  expected  to 
rise  as  technology  improves.  For  example,  insertion  of  pacemakers 
in  heart  disease  patients  may  be  appropriately  on  the  rise  as  the 
procedure  becomes  better  understood  and  practiced.  Since  it  is  also 
the  task  of  PSROs  to  identify  problem  areas  of  use  and  deal  with 
them  specifically,  potentially  problematic  surgical  procedures  and 
diagnostic  categories  were  identified  in  Use  Study  2,  and  examined 
to  determine  PSRO  effect  with  respect  to  each. 

Use  Study  2  attempted  to  control  heterogeneity  in  level  of 
review  by  using,  as  the  level  of  aggregation,  counties,  which  are  in 
general,  smaller  geographical  units  than  PSRO  areas.  There  should 
be  more  homogeneity  of  review  within  counties  than  within  PSROs. 
The  counties  are  then  aggregated,  for  analysis,  into  active  and 
inactive  PSRO  areas  based  on  the  level  of  review,  measured  as  the 
percent  of  all  area  Medicare  discharges  under  review. 

Because  a  certain  proportion  of  residents  of  PSRO  areas  are 
likely  to  receive  inpatient  care  in  other  areas,  use  rates  are 
difficult  to  calculate.  In  Use  Study  1,  a  "migration  factor" 
(discussed  in  Section  2.2.2.2)  was  used  to  adjust  enrollment  figures 
for  all  PSROs.  /1_  The  resulting  rates  are  provider-based  in  that 
the  number  of  discharges  within  a  PSRO  area  represent  the  number  of 
discharges  from  providers  within  that  area.  Use  Study  2,  on  the 
other  hand,  uses  beneficiary-based  rates  where  the  discharges 
for  any  county  represent  the  number  of  discharges  experienced  by 
residents  of  that  county  irrespective  of  where  they  occurred.  The 
latter  form  of  rate  calculation  more  nearly  relates  incidence  to  a 
given  population  and  is  thus  preferred  when  diagnosis-specific 
analyses  are  conducted.  Provider-based  rates  are  more  reflective  of 
provider  behavior  within  a  PSRO,  which  is  necessary  to  model  indi- 
vidual PSRO  effects. 

Finally  it  has  been  argued  that  PSRO  review  cannot  be  expected 
to  result  in  large  reductions  in  utilization  wherever  the  use  rates 
are  already  relatively  low.  In  other  words,  PSROs  should  have  the 
best  chance  of  effecting  reductions  where  overall  use  (including 
probable  inappropriate  use)  is  highest.  Acting  on  that  premise,  Use 
Study  2  stratifies  counties  into  high,  medium  and  low  use  groups  for 
each  diagnostic  category. 


/I  Deacon,  Ronald  and  James  Lubitz,  OPPR,  HCFA,  "Development  of 
Hospital  Utilization  Measurements  for  PSRO  Areas".  Presented 
at  the  Public  Health  Conference  on  Records  and  Statistics, 
Washington,  D.C.,  June  5,  1978. 
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Unlike  Use  Studies  1  and  2,  the  two  additional  studies  (Use 
Studies  3  and  4)  use  information  from  the  PSRO  Hospital  Discharge 
Data  Set  (PHDDS) .  The  PHDDS  contains  record  abstract  data  not 
only  on  Medicare  discharges  but  Medicaid  and  Title  V  (Maternal  and 
Child  Health)  discharges  as  well.  These  studies  represent  first 
uses  of  the  PHDDS  in  the  PSRO  context  and  are  thus  more  exploratory 
than  Use  Studies  1  and  2.  They  are  also  limited  by  the  fact  that 
non-PSRO  baseline,  or  comparison,  data  do  not  exist  since  only 
active  PSROs  collect  PHDDS  data.  Finally/  it  should  be  noted 
that  the  PHDDS  is  newly  operational:  data  for  1976  and  1977  were 
available  for  only  a  limited  number  of  PSROs.  While  it  is  expected 
that  analysis  of  PHDDS  data  will  assume  growing  importance  in  future 
PSRO  evaluations,  Studies  3  and  4  are  at  this  time  developmental  in 
the  sense  that  they  explore  the  potential  uses  of  the  PHDDS  for 
evaluation  purposes,  and  do  not  directly  attempt  to  draw  evaluative 
conclusions . 

Use  Study  3  attempts  to  examine,  longitudinally,  the  change  in 
ALOS  for  selected  diagnoses.  Pre-PSRO  use  data,  extracted  from  the 
Medicare  20  percent  sample  discharge  file  (the  same  data  base  used 
in  Use  Study  2),  is  compared  with  post-PSRO  use  data  derived  from 
the  PHDDS.  Clearly  there  are  potentially  severe  methodological 
problems  in  using  disparate  data  bases  for  a  pre-post  analysis.  The 
two  most  severe  difficulties  arise  from  differences  in  the  popula- 
tions covered  and  in  coding.  Diagnostic  codes  are  particularly 
subject  to  extreme  problems  of  abstracting  reliability.  /I 

Therefore,  the  first  part  of  the  study  was  devoted  to  a  method- 
ological comparison  of  utilization  by  diagnosis  from  both  the  20 
percent  sample  discharge  file  and  the  PHDDS  for  1976;  this  was  done 
for  six  PSROs  which  have  complete  PHDDS  data.  The  comparison  of 
these  two  files  showed  that  there  are  significant  differences 
between  the  two  data  sets  when  examining  diagnosis-specific  cate- 
gories. When  aggregated  across  all  diagnoses,  however,  the  two  data 
sets  are  quite  similar.  Therefore,  the  pre-post  analysis  is  limited 
to  those  hypotheses  which  are  not  diagnosis-specific.  The  specific 
hypotheses  tested  are  described  in  Section  2.4.3.1. 

Use  Study  4  is  an  examination  of  the  change  in  case  mix  sever- 
ity in  PSROs  over  time.  The  staging  methodology  is  used  to  measure 
the  severity  of  illness  in  selected  diagnostic  categories  in  active 
PSROs  over  time.  The  diagnoses  are  staged  with  respect  to  severity 
from  one  (least  severe)  to  three  (most  severe)  according  to  informa- 
tion available  in  the  discharge  abstract.  For  some  diagnostic 
categories ,  it  may  be  inappropriate  to  hospitalize  at  stage  1 . 
Therefore,    it  is  hypothesized  that  PSRO  review  has   resulted  in  a 


/I  Institute     of     Medicine,      Reliability  of  Hospital  Discharge  Ab- 
stracts, February,  1977,  Washington,  D.  C. 
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decrease  in  the  relative  percent  of  stage  1  admissions  for  those 
diagnostic  groups.  The  staging  methodology  is  also  used  to  evaluate 
changes  in  ALOS.  If  unnecessary  stage  1  cases  are  reduced,  the 
overall  ALOS  may  increase  (due  to  the  decline  in  admissions  of 
healthier  patients).  However,  the  length  of  stay  within  stages  of  a 
diagnostic  category  would  not  be  affected  by  changes  in  case  mix 
severity.  Therefore,  changes  in  ALOS  within  stages  provides  a 
superior  indication  of  PSRO  effectiveness  in  reducing  unnecessary 
days   of   care   than   do   changes    in  ALOS  across   all  stages  combined. 

Use  Study  4  results  are  necessarily  sensitive  to  the  validity 
of  the  PHDDS  data.  Changes  in  case  mix  indicating  a  less  healthy 
inpatient  population  after  PSRO  review  might  simply  reflect  better 
record  keeping  or  changes  in  recording  habits  of  physicians  rather 
than  the  effects  of  review.  Physicians  may  be  listing  more  compli- 
cations and  secondary  diagnoses  (factors  that  increase  a  patient's 
stage)  on  the  face  sheet  in  the  post-period  than  in  the  pre-period. 
A  longitudinal  study  (currently  underway)  of  discrepancies  between 
the  original  abstract  and  a  reabstract  of  the  medical  record  will 
test  this  competing  hypothesis. 

The  validation  aspect  of  Use  Study  4  is  not  as  relevant  to  this 
year's  evaluation  effort  as  it  is  in  improving  the  PHDDS  data  base 
for  future  use  in  evaluation  efforts.  Use  Study  4,  like  Use  Study 
3,  is  an  exploratory  study  using  the  PHDDS.  Much  of  the  information 
gleaned  from  these  two  studies  will  be  of  value  as  baseline  efforts 
for  the  future  evaluation  of  PSROs. 

2.1.3     Relationship  to  1977  Evaluation 

Use  Study  1  is  a  direct  extension  of  the  1977  Medicare  Utili- 
zation Study.  Essentially  the  same  data  bases  and  methods  are  used. 
However,  there  are  changes  in  this  year's  study  which  are  improve- 
ments upon  last  year's  analysis. 

The  1977  study  was  limited  to  eighteen  active  PSROs  which  had 
at  least  50  percent  of  area  Medicare  discharges  under  review  by  the 
first  quarter  of  1976;  while  24  PSROs  met  that  criterion,  six  had  to 
be  dropped  for  methodological  reasons.  For  each  active  area,  one  or 
two  comparison  areas  without  PSRO  review  were  selected  and  matched 
with  the  active  PSRO  areas  on  the  basis  of  geographic  region, 
demographic  characteristics,  and  characteristics  of  the  health  care 
delivery  system.  A  total  of  twenty-four  inactive  PSRO  areas  were 
used  for  comparison.  Despite  the  matching  of  active  and  inactive 
areas,  however,  the  active  PSROs  included  in  the  analysis  did  not 
constitute  a  representative  sample  of  the  universe  of  active  PSROs. 
Consequently,  statistical  extrapolation  of  the  1977  study  findings 
to  the  PSRO  program  nationally  was  not  warranted.  In  1978,  however, 
all  PSRO  areas,  active  and  inactive,  are  included  in  the  analysis. 
Thus,  the  1978  evaluation  results  are  more  representative  of  the 
national  PSRO  program  than  were  the  1977  findings. 
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The  PSRO  program,  as  of  the  1977  evaluation,  was  still  very 
new.  Only  twenty-four  PSROs  had  attained  a  50  percent  level  of 
Medicare  discharge  review  by  the  first  quarter  of  1976.  By  the 
first  quarter  of  1977,  however,  65  PSROs  had  attained  this  level  of 
review.     Clearly,  the  program  is  rapidly  maturing. 

The  1977  Medicare  Utilization  Study  incorporated  a  number  of 
adjustor  variables  to  control  for  the  health  resource  and  demo- 
graphic characteristics  of  active  and  inactive  PSRO  areas  which 
might  have  an  effect  on  hospital  utilization  independent  of  the 
impact  of  PSRO  review.  Among  these  variables  were  population 
density,  hospital  occupancy  rate,  and  percent  of  population  with 
less  than  $5,000  income.  Other  adjustor  variables  used  this  year 
are  briefly  described  below: 

1 .  The  ability  of  hospitals  to  discharge  long  stay  patients  to 
nursing  homes  can  be  limited  if  nursing  home  beds  are 
unavailable.  Accordingly,  nursing  home  beds  per  1,000 
population  was   added  as   an   adjustor   variable   this  year. 

2 .  Since  the  supply  characteristics  of  an  area  can  affect  its 
use  rates,  physicians/1,000  population  and  short-stay 
hospital  beds/1, 000  population  were  also  added. 

3.  In  addition,  supplementary  analyses  were  conducted  to  test 
the  hypotheses  that  PSRO  impact  is  affected  by  regional 
characteristics  and  by  rate  setting  commissions. 

Another  characteristic  of  the  1977  Medicare  Utilization  Study 
is  its  use  of  utilization  rates  based  on  the  ratio  of  hospital 
DISC  and  DOC  within  a  PSRO  area  to  Medicare  enrollment  within  that 
area.  At  that  time,  it  was  not  possible  to  adjust  for  the  fact  that 
some  percentage  of  the  utilization  within  a  PSRO  is  made  by  Medicare 
eligibles  residing  in  other  PSROs.  Use  Study  1  controls  for  patient 
migration  by  adjusting  enrollment  figures  to  reflect  net  in-flow  and 
out-flow  of  patients  between  PSROs. 

The  1977  Medicare  Utilization  Study  failed  to  find  a  PSRO 
program  impact  on  utilization.  Nevertheless,  the  possibility 
remained  that  the  program  may  have  reduced  utilization  in  certain 
diagnostic  categories.  Part  of  the  rationale  behind  Use  Study  2  is 
to  test  this  hypothesis.  The  data  used  in  this  study  are  1976 
Medicare  discharge  data,  the  same  year  used  in  the  1977  Medicare 
Utilization  Study.  Therefore,  if  no  program  effect  is  demonstrated, 
Use  Study  2  will  suggest  that  not  only  was  there  no  program  effect 
on  all  discharges  collectively,  there  was  no  discernible  effect  on 
selected  individual  diagnoses  as  well.     If  a  program  effect  is_  found 
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for  some  diagnoses,  it  would  suggest  that  PSROs  did  indeed  have  an 
impact  on  utilization  as  of  1976,  although  it  was  not  measurable  at 
the  aggregate  level. 

The  1977  evaluation  had  only  one  study  (a  staging  analysis) 
which  examined  diagnosis-specific  discharge  data;  the  1978  evalua- 
tion has  three  (Use  Studies  2  through  4).  Although  the  data, 
methodology,  and  sample  of  PSRO  areas  differ  for  each  study,  the 
1978  evaluation  benefits  by  having  three  independent  studies  of  the 
effect  of  PSRO  review  on  diagnosis-specific  indicators. 

Finally,  the  1977  evaluation  included  a  study  of  hospital 
discharge  abstracts  which  analyzed  ALOS  and  case  mix  severity  as 
proxied  by  the  staging  methodology.  Discharges  from  95  hospitals 
which  had  begun  Medicare-binding  PSRO  concurrent  review  between 
September  1974  and  January  1,  1976  were  compared  with  discharges 
from  a  random  sample  of  5  2  hospitals  not  performing  PSRO  review./ 1 
PSRO  and  non-PSRO  areas  were  compared  with  respect  to  changes  in 
ALOS,  percent  of  long-stay  and  short-stay  patients  and  illness 
severity. 

Use  Study  4  does  not  directly  replicate  the  1977  staging 
analysis  but  rather  applies  the  staging  methodology  to  a  different 
data  set,  the  PHDOS.  Although  no  comparison  is  possible  between 
active  and  inactive  PSRO  areas,  longitudinal  information  is  gathered 
on  a  limited  number  of  PSRO  areas. 

2.1.4  Limitations 

The  limitations  and  qualifying  statements  relevant  to  each  of 
the  four  studies  are  discussed  in  greater  detail  in  the  individual 
sections.  However,  there  are  several  limitations  in  data  model 
specification  and  analysis  which  are  common  to  most  of  the  studies. 

Use  Studies  1  and  2  use  Medicare  data  to  evaluate  the  PSRO 
program.  These  studies  cannot  answer  the  question  of  PSRO  effec- 
tiveness with  regard  to  Medicaid  and  Title  V  discharges,  which  also 
come  under  PSRO  review.  While  the  PHDDS,  which  includes  Medicaid 
and  Title  V  discharges,  is  used  in  Use  Studies  3  and  4,  only  Study  4 
actually  examines  the  Medicaid  data.  In  that  study,  however,  there 
is  no  comparison  group  (inactive  PSROs)  for  Medicaid  patients  and 
DOC  or  DISC  rate  calculations  are  not  made.  Therefore,  there  are  no 
estimates  of  PSRO  impact  on  Medicaid  and  Title  V  patients  in  this 
1978  evaluation. 


/1  Both  PSRO  and  non-PSRO  hospitals  were  selected  from  one  of 
two  abstracting  services,  the  Commission  on  Professional  and 
Hospital  Activities  and  the  California  Health  Data  Corporation. 
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Many  factors  constitute  competing  explanations  for  observed 
changes  in  social  phenomena.  Use  Studies  1  and  2  attempt  to  adjust 
for  factors  which  may  affect  change  in  use  rates.  However,  there  is 
always  the  chance  that  some  relevant  characteristics  have  not  been 
considered  or  that  the  analysis  itself  does  not  properly  model  the 
real  world  effects.  Further,  the  self -selection  of  PSROs  and 
hospitals  within  PSROs  in  the  active  group  is  always  a  potential 
problem. 

The  level  of  aggregation  also  varies  among  the  studies.  Use 
Study  1  examines  aggregate  ALOS,  DOC  and  DISC  rates  whereas  Use 
Studies  2  through  4  are  diagnosis-specific.  While  Use  Study  2 
examines  ALOS  and  DISC,  Use  Studies  3  and  4  are  limited  to  ALOS 
analyses.  Despite  the  possibility  that  differing  results  from 
disparate  data  bases  could  provide  interpretive  difficulties,  the 
study  results  were  for  the  most  part  found  to  be  compatible. 

The  studies  incorporate  several  measures  of  program  charac- 
teristics. The  primary  comparison  is  between  active  and  inactive 
PSROs.  Active  PSROs  are  classified  according  to  level  of  review, 
period  of  operation  and  delegation  status.  Other  factors  which 
might  operate  within  a  PSRO  to  effect  use  are  not  considered  by  the 
1978  utilization  impact  studies  since  they  could  not  be  properly 
quantified.  A  careful  linking  of  program  characteristics  to  impact 
results  has  yet  to  be  accomplished. 

A  final  consideration  is  perhaps  the  most  difficult  to  evalu- 
ate. Use  Studies  1  and  2  depend  upon  the  selection  of  active  and 
inactive  PSROs  and  their  subsequent  comparison.  To  the  extent  that 
Utilization  Review  (UR)  activity  was  conducted  in  the  inactive 
areas,  the  studies  are  in  effect  comparing  one  form  of  UR  with 
another.  The  contrast  is  thus  between  PSRO  UR  and  hospital-based 
UR;  as  a  consequence  the  evaluation  actually  estimated  the  incre- 
mental PSRO  impact  over  and  above  that  of  hospital-based  UR.  To  the 
extent  that  hospital-based  UR  has  not  been  particularly  successful, 
the  estimates  of  incremental  PSRO  impact  approximate  the  total 
impact  of  program  review  activities.  For  cost-benefit  and  program 
evaluation  purposes,  however,  incremental  PSRO  program  impact  is  the 
preferred  variable.  This  is  especially  true  for  disaggregated 
analyses  which  compare  one  PSRO  to  another. 

2.2    One  Hundred  Percent  Medicare  Claims  File  Study 

2.2.1  Objectives 

This  study  seeks  to  determine  if  areas  with  active  PSROs 
experience  a  decline  (or  less  rapid  increase)  in  overall  rates  of 
hospitalization  in  comparison  to  inactive  PSRO  areas.  The  major 
focus  of  the  analysis  is  on  Medicare  days  of  care  rates  (DOC)  since 
this  variable   is  the  best  measure  of  the  primary  goal  of  reducing 
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inappropriate  utilization  of  hospital  services  and  thus  reducing  the 
rate  of  increase  of  hospital  expenditures.  Other  variables  such  as 
discharge  rates  (DISC),  average  length  of  stay  (ALOS),  and  the 
percent  of  short  and  long  stays,  are  included  in  the  evaluation 
since  they  may  be  useful  for  interpreting  changes  which  occur  in 
DOC.  Given  the  various  objectives  of  PSROs,  measures  such  as  DISC 
or  ALOS  may  be  influenced  in  either  an  upward  or  a  downward  direc- 
tion (or  not  influenced  at  all).  For  example,  a  PSRO  which  focuses 
on  eliminating  unnecessary  short  hospitalizations  may  experience  an 
increase  in  the  average  length  of  stay,  whereas  a  PSRO  which  focuses 
on  the  problem  of  moving  long-stay  patients  to  more  appropriate  care 
facilities  may  see  a  decline.  Averaged  over  a  number  of  differently 
functioning  PSROs,  a  variable  such  as  ALOS  could  appear  to  be 
uninfluenced  by  PSRO  implementation  status.  However,  the  total 
number  of  days  of  care  rendered  should  decline,  regardless  of 
individual  PSRO  characteristics. 

In  addition  to  reducing  DOC,  a  favorable  impact  due  to  PSRO 
review  would  be  an  expected  reduction  in  the  percent  of  long  hos- 
pitalizations. Thus,  long  stays  and  short  stays,  expressed  as 
percents  of  all  stays,  are  included  among  the  outcome  variables  for 
comparative  purposes. 

PSROs  may  conduct  their  own  review  or  delegate  review  to  the 
hospitals.  Within  a  given  area,  some  hospitals  might  conduct  their 
own  review  while  the  PSRO  conducts  the  review  for  other  hospitals. 
The  percent  of  discharges  under  delegated  review  is  a  measure  of  the 
extent  to  which  the  review  process  is  under  direct  PSRO  control.  It 
could  be  argued  that  this  delegation  process  might  have  an  effect  on 
PSRO  impact;  that  is,  the  greater  the  percent  delegation,  the  less 
effective  the  PSRO.    This  hypothesis  was  among  those  tested. 

The  primary  study  hypotheses  are  as  follows: 

(1)  There  has  been  a  reduction  in  DOC  in  active  PSRO  areas 
relative  to  inactive  areas. 

(2)  There  has  been  a  reduction  in  DISC  in  active  PSRO  areas 
relative  to  inactive  areas. 

(3)  There  has  been  a  reduction  in  ALOS  in  active  PSRO  areas 
relative  to  inactive  areas. 

(4)  Within  active  PSRO  areas,  the  percent  of  discharges  under 
delegated  review  is  inversely  correlated  with  PSRO  effec- 
tiveness. 

In  addition,  secondary  research  questions  were  tested  in 
supplementary  analyses.    These  questions  were  as  follows: 


51 


(1)  What  has  been  the  impact  of  individual  PSROs  on  DOC? 

(2)  Are  there  regional  differences  in  PSRO  impact? 

(3)  Do  PSROs  become  more  effective  with  time? 

(4)  Do  hospital  rate  review  activities  impair,  increase,  or 
have  no  effect  on  PSRO  impact? 

2.2,2  Data 

2.2.2.1    Data  Sources 

( 1 )  100  Percent  Medicare  Claims  File:  Every  hospital  admis- 
sion for  which  a  Medicare  claim  is  made  is  entered  into 
this  file.  It  is  a  complete  count  of  discharges  and  days 
of  care.  The  address  of  the  provider  is  also  included  so 
that  discharges  and  days  of  care  can  be  aggregated  to  the 
PSRO  level.  Data  were  used  for  the  years  1974,  1975, 
1976,  and  1977  for  which  there  were  7.1  million,  7.3 
million,  7.7  million,  and  8.1  million  acute  care  dis- 
charges of  aged  Medicare  enrollees  respectively. 

(2)  The  Medicare  Master  Enrollment  File;  This  file  contains 
records  for  all  Medicare  enrollees  in  the  United  States. 
The  use  of  this  file  and  the  100%  Claims  File  enables 
the  calculation  of  rates.  The  number  of  aged  Medicare 
enrollees  in  1974,  1975,  1976  and  1977  were  22.0  million, 
22.5  million,  22.9  million,  and  23.5  million  respectively. 

(3)  The  Medicare  Provider  of  Service  File;  This  file  contains 
information  on  inpatient  providers,  i.e.,  hospitals 
qualified  for  Medicare  reimbursement.  By  examining  the 
date  when  each  provider  came  under  binding  PSRO  review  it 
is  possible  to  estimate  the  level  of  review  (percent  of 
all  discharges  under  review)  for  any  PSRO  area  for  any 
time  period  in  the  study. 

(4)  The  Master  Facility  Inventory:  This  data  base,  provided 
by  the  National  Center  for  Health  Statistics,  contains 
important  information  on  supply  factors  (number  of  general 
hospital  beds  and  nursing  home  beds)  which  may  affect 
Medicare  utilization  rates  independently  of  any  PSRO 
effect. 

(5)  The  Area  Resource  File;  This  data  base,  maintained  by  the 
Bureau  of  Health  Manpower,  contains  social  and  medical 
environmental  factors  which  also  need  to  be  controlled  for 
in  the  analysis.  These  factors  include  population  den- 
sity, number  of  families  with  yearly  incomes  below  $5,000, 
and  number  of  physicians/1000  population. 
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2.2.2.2    Data  Adjustments 


(1)  Semi -Annual  Enrollment  Estimates:  In  order  to  calculate 
semi-annual  use  rates  it  was  necessary  to  derive  estimates  of  semi- 
annual enrollment  rates.  Actual  counts  of  Medicare  enrollees  were 
available  for  January  1  and  July  1  of  each  year.  It  was  decided 
that  enrollment  counts  for  the  midpoint  of  each  semi-annual  period 
(April  1  and  October  1),  would  be  the  best  estimate  for  the  calcu- 
lation of  rates.  The  midpoints  were  calculated  by  using  an  algo- 
rithm which  takes  into  account  the  fact  that  the  growth  of  Medicare 
enrollment  varies  throughout  the  year./J^  For  instance,  the  growth 
rate  slows  in  the  first  quarter  of  the  year  (January  through  March) 
because  the  mortality  rate  for  Medicare  enrollees  is  higher  at  this 
time. 

(2)  Enrollment  Adjustments  for  Migration;  The  utilization 
rates  were  adjusted  for  migration  of  Medicare  enrollees  from  the 
PSRO  area  of  residence  to  another  PSRO  area  for  the  receipt  of 
health  care.  A  detailed  description  of  this  procedure  is  provided 
by  Deacon  and  Lubitz./2       This  section  summarizes  the  method. 

Use  rates  in  the  1977  PSRO  Evaluation  were  calculated  by 
dividing  the  number  of  Medicare  discharges  (or  days  of  care) 
occurring  in  hospitals  in  a  PSRO  area  by  the  number  of  Medicare 
enrollees  residing  in  that  area.  If  patients  were  hospitalized 
only  in  hospitals  in  their  PSRO  area  of  residence,  this  method 
would  yield  a  perfectly  valid  rate.  However,  because  up  to  63%  of 
patients  cross  PSRO  boundaries  to  receive  care,  there  is  a  lack  of 
correspondence  between  numerator  and  denominator  in  this  rate 
calculation.  This  lack  of  correspondence  can  lead  to  a  serious 
miscalculation  of  use  rates  for  the  population  actually  at  risk. 
Because  PSROs  are  responsible  for  monitoring  care  of  the  providers 
in  their  areas,  the  numerator  (discharges  from  area  hospitals)  seems 
reasonable.  The  task,  then,  is  to  adjust  the  denominator  to  reflect 
the  population  at  risk  rather  than  the  area  population. 

Using  an  adaptation  of  a  method  proposed  by  Bailey, /3  the 
population   at   risk   for   a   given  PSRO  was   estimated  by  allocating 


/ 1  Baugh,  David,  "1978  PSRO  Evaluation  Data  Base,  Data  Sources, 
Techniques  and  Quality."  HCFA,  OPPR,  Office  of  Statistics  and 
Data  Management. 

/2  Deacon  and  Lubitz,  Op.  cit. 

/3  Norman  T.  J.  Bailey,  "Statistics  in  Hospital  Planning  and  De- 
sign," Applied  Statistics,  November  1965,  pp.  146-157. 
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portions  of  Medicare  enrollment  from  all  PSRO  areas  based  on  each 
individual  PSRO's  contribution  (as  a  percent  of  its  total  dis- 
charges) to  the  patient  load  in  the  given  PSRO  area.  For  example, 
if  25  percent  of  the  discharges  of  PSRO  A's  enrollees  occurred  in 
PSRO  B,  then  25  percent  of  PSRO  A's  enrollment  would  be  allocated  to 
PSRO  B.  Similarly,  if  10  percent  of  discharges  of  PSRO  B  enrollees 
occurred  in  PSRO  A,  then  10  percent  of  PSRO  B's  enrollment  would  be 
allocated  to  PSRO  A.  This  algorithm  was  applied  to  all  PSROs. 
Therefore,  the  adjusted  enrollment  for  each  PSRO  is  a  net  change 
reflecting  both  the  out-flow  of  patients  to  all  other  PSROs  and  the 
in-flow  of  patients  from  all  other  PSROs.  This  adjustment  was  done 
for  years  1974,  1975,  1976,  and  1977. 

(3)  Adjustment  Variables  Presented  as  Change  Variables; 
Because  the  analysis  described  below  is  essentially  a  rate  of  change 
model,  it  is  appropriate  that  adjustment  variables  be  entered  into 
the  analysis  to  reflect  changes  in  the  adjustment  variables  from  the 
base  period  (1974)  to  the  most  recent  observation  period  (1977). 
Two  problems  exist  with  this  theoretical  notion.  First,  1977  data 
are  not  yet  available  for  most  of  the  adjustment  variables. 
In  general,  however,  1976  data  was  available,  so  adjustment  vari- 
ables could  be  calculated  to  reflect  changes  from  1974  to  1976.  A 
more  significant  problem  is  that  changes  in  certain  adjustment 
variables  might  correlate  with  the  impact  of  PSRO  review.  If  this 
were  true,  the  inclusion  of  these  variables  in  change  form  could 
tend  to  reduce  the  apparent  effect  of  PSRO  review.  This  argument 
applies  most  particularly  to  percent  of  total  caseload  due  to 
Medicare  and  to  the  occupancy  measure.  For  example,  if  PSRO  review 
were  to  reduce  Meidcare  DOC  within  a  given  PSRO,  we  would  simul- 
taneously expect  a  decline  in  the  occupancy  rate  and  a  smaller 
proportion  of  discharges  due  to  Medicare  (assuming  all  other  factor?? 
remained  constant).  While  the  actual  influence  on  these  measures 
might  not  operate  in  such  a  simplistic  manner  (e.g.,  some  beds  might 
be  eliminated  thereby  restoring  the  original  occupancy  rate)  there 
is  nonetheless  the  potential  for  positive  PSRO  impact  to  be  re- 
flected in  changes  in  these  adjustment  variables.  For  this  reason, 
both  the  dependent  variable  Medicare  DOC  (or  DISC  in  the  discharge 
rate  analysis)  and  these  two  adjustment  variables  —  occupancy  rate 
and  percentage  of  caseload  due  to  Medicare  —  were  entered  in  terms 
of  their  latest  values  (1976).  On  the  other  hand,  the  four  adjust- 
ment variables  related  to  physicians,  nursing  home  beds,  short-stay 
hospital  beds,  and  proportion  of  aged  population  were  entered  in 
change  form,  i.e.  the  numeric  difference  between  the  value  of  that 
variable  in  1976  and  its  value  in  1974  ( 1973  in  the  case  of  nursing 
home  beds).  The  specific  adjustment  variables  used  in  the  analysis 
are  identified  in  Table  8. 
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TABLE  8 


ADJUSTMENT  VARIABLES 


Variable 
Population  density 

Proportion  of  families  under  $5,000  income 

/1,000  population 
(Change)  physicians/1,000  population 
(Change)  short-stay  beds/ 1,000  population 
(Change)  nursing  home  beds/1, 000  population 
(Change)  percent  of  total  population  age  65 

and  over 
Occupancy  rate 

Percent  of  caseload  due  to  Medicare 


Year ( s ) 

1970 

1976 
( 1976)-( 1974) 
(1976)-(1974) 
(1976)-( 1973) 

(1976)-(1974) 
1976 
1976 


Source/ 1 

ARF 

ARF 
ARF 
ARF 
MFI 

Medicare 
* 

** 


/1  ARF  =  Area  Resource  File,  Bureau  of  Health  Manpower 

MFI  =  Master  Facility  Inventory,  National  Center  for  Health 
Statistics 

*  =  Calculated  from  MFI  data;  occupancy  =  total  inpatient 
days  of  care/(365  x  number  of  general  hospital  beds) 
**  =  Calculated  from  Medicare  and  MFI  data;  discharge  case- 
load =  Medicare  discharges/all  discharges;  DOC  case- 
load =  Medicare  DOC/all  DOC. 
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2.2.3  Methods 

2.2.3.1    Definition  of  Active  and  Inactive  Areas 

The  issue  of  how  to  categorize  PSROs  into  active  and  inactive 
groups  is  not  simple.  There  are  degrees  of  implementation  status  at 
any  one  point  in  time.  In  addition,  PSROs  change  over  the  course  of 
a  year.  Thus,  one  PSRO  may  start  review  in  the  first  quarter  of 
1977  and  review  only  5  percent  of  discharges.  By  the  fourth  quarter 
of  1977  that  PSRO  may  have  attained  a  review  level  of  100  percent. 
Another  PSRO  may  have  a  constant  rate  of  review  throughout  the  year 
of  50  percent.  In  terms  of  degree  of  implementation  these  two 
hypothetical  PSROs  have  changed  position  from  the  first  to  second 
half  of  the  year.  Yet  their  overall  review  rate  for  the  entire  year 
may  be  the  same. 

It  was  decided  that  only  PSROs  that  had  begun  binding  review 
as  of  April  1,  1977  would  be  considered  as  active  PSROs.  This  would 
insure  that  they  had  been  active  at  least  three-fourths  of  the  year. 
Ninety-six  PSROs  met  this  criterion.  Another  criterion  originally 
considered  was  to  define  as  active  only  those  PSROs  that  had  a- 
chieved  a  50%  level  of  review  by  April  1,  1977.  This  criterion, 
used  in  the  1977  report,  was  met  by  only  sixty-five  PSROs.  The 
rationale  for  its  use  was  that  PSROs  might  be  expected  to  affect 
overall  utilization  only  when  they  had  a  majority  of  discharges 
under  review. 

If  the  50  percent  criterion  is  applied  to  the  1978  evaluation 
data,  however,  a  problem  arises  as  to  what  to  do  with  the  thirty-one 
PSROs  which  had  some  review  but  not  50  percent.  If  included  with 
the  inactive  group  they  will  bias  the  results;  that  is,  to  the 
extent  that  these  PSROs  have  lowered  utilization,  they  will  lower 
the  overall  inactive  rate  and  obscure  the  difference  between  active 
and  inactive  areas.  A  second  alternative  is  to  exclude  the  thirty- 
one  "partially  active"  PSROs  from  the  analysis  altogether.  This, 
however,  would  neglect  to  use  what  could  be  valuable  data.'  New 
PSROs  may  have  characteristics  or  changes  in  utilization  that  are 
important  to  document. 

An  inspection  of  the  data  for  the  thirty-one  partially  active 
PSROs  indicates  that  they  had  achieved  a  significant  level  of  review 
(46%)  over  1977.  While  not  as  high  as  that  for  the  sixty-five  more 
fully  active  PSROs  (93%),  it  is  large  enough  to  assume  that  PSRO 
review  activity  might  have  had  an  effect.  By  specifying  the  regres- 
sion analyses  to  test  for  PSRO  impact  based  on  intensity  (level  of 
review)  and  longevity  (length  of  time  review  has  been  implemented), 
continuous  measures  of  program  activity  can  be  used  to  explain 
variations  in  program  impact.  This  specification  allows  for  the 
inclusion  of  less  fully  implemented  PSROs  without  "penalizing"  the 
remaining  active   PSROs   by    reducing   the   overall   estimate   of  PSRO 
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impact.  Given  the  relatively  high  degree  of  implementation  of  the 
thirty-one  partially  implemented  PSROs  and  the  use  of  continuous 
measures  of  program  activity,  it  was  decided  that  all  ninety-six 
PSROS  which  had  begun  binding  review  by  April,  1977  would  be  class- 
ified as  active. 

2.2.3.2  Descriptive  Analyses 

This  report  includes  descriptive  analyses  (Section  2.2.4.1) 
which  present  the  reader  with  the  unadjusted  data  and  thereby 
provide  a  "feel"  for  use  rates  and  adjustor  variable  characteristics 
in  active  and  inactive  PSROs.  Utilization  rates  are  presented 
graphically  to  show  changes  over  time.  A  statistical  comparison  of 
active  and  inactive  PSROs  with  regard  to  adjustor  characteristics  is 
also  presented. 

2.2.3.3  Regression  Analyses 

Regression  analysis  was  the  primary  analytical  technique 
employed  in  Use  Study  1 .  Numerous  regression  equations  were  esti- 
mated, all  of  which  lend  a  slightly  different  perspective  to  the 
analysis.  Common  to  all  of  these  estimated  equations,  however,  was 
a  basic  regression  structure  which  was  predicated  on  five  funda- 
mental analytical  considerations. 

First,  the  model  specification  involves  a  selection  of  outcome 

variables   which    serve   as    dependent   variables   for   the  regression 

analysis.  The  outcome  variables  selected  are  those  previously 
mentioned  in  Section  2.2.1. 

Second,  the  focus  was  on  the  change  in  utilization  rates  from 
the  baseline  period  (1974)  to  the  target  year  (1977).  Thus,  the 
analysis  must  take  into  account  and  adjust  for  the  base  measures  of 
each  dependent  variable.  1974  was  chosen  as  the  base  year  because 
this  is  the  earliest  pre-PSRO  program  year  for  which  data  exist. 
Later  years  are  contaminated  with  "transition"  effects  associated 
with  the  development  of  the  PSRO  program. 

Third,  the  analysis  must  adjust  for  those  variables  which  might 
conceivably  affect  PSRO  impact.  Unless  this  is  done,  the  estimated 
impact  of  PSRO  review  may  be  confounded  with  one  or  more  of  those 
factors. 

Fourth,  the  regression  model  must  specify  the  form  of  the 
program  variable  intended  to  index  the  impact  of  PSRO  review.  The 
regression   models   use  three   such  program  variable  specifications. 

Fifth,  program  impact  may  vary  with  either  the  base  or  adjust- 
ment variables  in  a  manner  which  produces  unique  interactive  ef- 
fects.    The  analysis  must  permit  assessment  of  those  effects. 

These  issues  are  discussed  below. 
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Indices  of  PSRO  Program  Activity:  The  simplest  form  uses  a 
program  "dummy."  This  variable  takes  the  value  1  if  an  area  is  an 
active  PSRO  (as  previously  defined)  and  0  otherwise.  The  regression 
coefficient  associated  with  this  variable  suggests  the  degree  to 
which  active  PSROs  differ  from  inactive  PSROs  for  any  given  outcome 
measure  after  controlling  for  base  and  other  adjustment  variables. 

The  other  two  measures  of  PSRO  program  activity  are  the  con- 
tinuous variables  intensity  and  longevity.  Intensity  is  measured  as 
the  percent  of  all  Medicare  discharges  under  PSRO  review;  longevity 
is  measured  as  the  number  of  months  since  the  first  hospital  came 
under  review.  One  would  expect  that  a  PSRO  with  a  high  level  of 
intensity  (90  to  100  percent  of  discharges  under  review)  would  have 
a  greater  impact  than  one  with  a  lower  level  of  intensity  (50 
percent  or  less).  Similarly,  a  PSRO  with  twenty-four  months  of 
review  experience  may  be  expected  to  be  more  effective  than  one  with 
only  six  months  of  experience. 

Forced-order  Multiple  Regression  Analysis;  This  technique 

provides  an  approach  to  estimating  and  testing  the  significance  of 
effects  while  meeting  the  considerations  raised  above.  In  general, 
the  technique  involves  the  following:  for  a  given  outcome  variable 
(e.g.,  total  days  of  care  rate),  sets  of  independent  variables 
(base,  adjustment,  program,  and  interaction)  are  entered  in  a 
predetermined  order  to  build,  in  a  stepwise  fashion,  a  multiple 
regression  equation  for  explaining  variations  in  the  outcome  vari- 
able. Tests  of  significance  are  made  at  each  stage  of  analysis  so 
that  the  significance  of  the  incremental  contribution  of  each  set  of 
variables  can  be  assessed.  The  interaction  variables  are  formed  by 
multiplying  the  program  variable  by  the  base  and  adjustment  vari- 
ables. 

This  is  a  conservative  approach  in  that  the  test  of  signifi- 
cance for  the  PSRO  variable  is  performed  only  after  the  regression 
equation  has  controlled  for  alternative  explanatory  (adjustment) 
variables. 

The  Primary  Regression  Specification:  All  of  the  regression 
specifications  take  the  general  form: 

(1)     Y  =  a  +        X,  +  b2  X2  +  b3  X3  .   .  .   .  +  b2Q  X2Q  +  e 

where : 

Y  =  a  vector  of  observations  taken  from  1977  data  (n=189) 
on  one  of  the  following  dependent  variables: 

•  Days  of  care  per  1000  Medicare  enrollees  (DOC); 
live  and  total;  annual  and  semi-annual. 

•  Discharges  per  1000  Medicare  enrollees  (DISC); 
live  and  total;  annual  and  semi-annual. 
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•  Average  length  of  stay  (ALOS); 

live  and  total;  annual  and  semi-annual. 

•  Percent  of  total  discharges  which  have  a 
length  of  stay  of  1  to  3  days;  annual. 

•  Percent  of  total  discharges  which  have  a 
length  of  stay  of  22  or  more  days;  annual. 

a  =  the  constant  term  in  the  regression 

b^,  b^g  =  unstandardized  regression  coefficients 

corresponding  to  the  independent  variables 
defined  below: 

Variable 

Variable  Name  Stage 

base  value  for  the  dependent  BASE  1 

variable  under  consideration 
(e.g.,   1974  DOC) 

population  density  POP  2 

proportion  of  population  POVERTY  2 

<  $5,000  family  income 

Physicians/ 1 ,000  population  MDs  2 

Short-stay  beds/1, 000  population    BEDS  2 

Nursing  heme  beds/1, 000  popu-  NURS  2 

lation 

Percent  of  population  who  are         AGED  2 
aged 

Occupancy  rate  in  short-stay  OCC  2 

hospitals 

Percent  of  caseload  due  to  CASELOAD  2 

Medicare 

Program  variable  (i.e.,  dummy ,      PROGRAM  3 
longevity,  or  intensity) 

Delegation  status  (used  only  -  4 

with  intensity) 

BASE  X  PROGRAM  -  5 
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Variable 


Variable  Name  Stage/ 1 

X13  =  POP  X  PROGRAM  -  5 

X. „  =  POVERTY  X  PROGRAM  -  5 
14 

X,r  =  MDS  X  PROGRAM  -  5 
1  b 

X.c  =  BEDS  X  PROGRAM  -  5 
1  b 

X1?  -  NURS  X  PROGRAM  -  5 

X18  =  AGED  X  PROGRAM  -  5 

X      =  OCC  X  PROGRAM  -  5 

X-.  ■  CASELOAD  X  PROGRAM  -  5 


e  =*  the  error  term 


/1  Stage  indicates  the  order  in  which  the  variables  are  entered  into 
the  equation.  In  stage  1 ,  the  base  rate  is  entered ,  stage  2 
enters  the  adjustment  variables,   stage  3  the  program  variables 

and  so  on. 

The  statistical  test  for  the  Increase  in  explained  variance  at 
stage  i  is  given  by  an  F  statistic: 

F±  =  [(R*  -  R^/Pil*  Id  "  R4)/  (n-p-1)] 

2  2  2         2  2 

where  R^  is  R    for  stage  i  (and  R^  -  R^-j  is  the  -increase  to  R 

at  stage  i),  p^  is  the  number  of  independent  variables  entered 
at  stage  i,  n  is  the  total  sample  size,  and  p  is  the  total  mnber 
of  independent  variables  entered  at  all  stages.  is  distrib- 

uted as  a  Snedecor  F  statistic  with  p^  and  n  -  p  degrees  of  free-* 
dan ,  (This  F  statistic  also  tests  the  significance  of  individual 
regression  coefficients.)  Only  if  F^  is  statistically  signifi- 
cant (at  least  at  the  .05  level)  for  a  particular  stage  (e.g., 
for  the  adjustment  variables  entered  at  stage  2)  is  it  worthwhile 
to  assess  their  separate  contributions  to  predicting  the  outcome 
measure.  The  major  focus  of  analysis  is  on  stage  3  which  pro- 
vides the  estimate  of  program  impact  on  the  designated  outcome 
measure. 
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If  there  are  no  significant  interactions,  the  general  form  of 
equation  (1)  can  be  summarized  as: 

(2)  Y(  1977)  =  a  +  b  X  ( 1974)  +     T.  b  •  (adjustment  variable  i) 

ii  i 

+  b10X10  +  e 

where  is  the  program  variable  and  b^  the  estimate  of  its 

impact. 

This  form  is  equivalent  to: 

(3)  Y(1977)   -  b1X1(1974)   =  a  +     £  b .  (adjustment  variable  i) 

+b10X10  +e 

The  left  hand  side  of  equation  (3)  is  the  difference  between 
the  1977  value  of  the  dependent  variable  and  the  1974  value  adjusted 
by  b^.  This  new  dependent  variable  (AY)  is  a  function  of  the 
adjustment  variables,  the  program  variable,  plus  a  constant  and  an 
error  term.  The  use  of  the  program  variable  provides  the  desired 
estimate  of  program  impact.  This  model  is  conceptually  equivalent 
to  measuring  the  difference  between  active  and  inactive  PSRO  areas 
with  respect  to  changes  in  the  dependent  variable  from  1974  to  1977 
while  controlling  for  all  other  variables  which  might  plausibly 
contribute  to  impact. 

The  reason  that  a  simple  difference  (i.e.,  Y^^^  -  Y1974^ 
was  not  used  as  the  dependent  variable  in  the  original  specification 
is  that  a  simple  difference  does  not  take  into  account  the  potential 
correlation  between  the  base  value  (1974)  and  the  observed  differ- 
ence (1977-1974);  i.e.,  the  lower  the  base,  the  smaller  the  change 
is  likely  to  be.  By  using  the  base  value  as  an  adjustor,  this 
slight  bias  is  removed. 

Disaggregated  Analysis  Based  on  the  Primary  Regression  Specifi- 
cations :  The  estimate  of  reduction  in  DOC  due  to  PSRO  review 
involves  the  calculation  of  two  "expected"  DOC  rates :  ( 1 )  the 
expected  non-PSRO  DOC  and  (2)  the  expected  PSRO  DOC.  The  difference 
between  these  values  is  the  estimated  program  impact.  Using  the 
following  definitions,  the  calculation  is  shown  below: 

ENP  =  the  expected  1977  DOC  had  the  PSRO  not  been  operating 

OR  =  the  observed  (or  actual)  1977  DOC 

EP  =  the  expected  1977  DOC  given  that  the  PSRO  was  operative 
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TD  =  the  total  (or  gross)  difference  between  ENP  and  OR 
PE  =  the  PSRO  effect 

RD  =  the  residual  difference  between  TD  and  PE,  i.e.,  that 
portion  of  TD  which  is  not  explained  by  PSRO  impact. 

By  definition: 

(4)  PE  =  TD  -  RD 
Note  that: 

(5)  TD  =  ENP  -  OR 

and 

(6)  RD  =  EP  -  OR 
Substituting  (5)  and  (6)  into  (4): 

(7)  PE  =  (ENP  -  OR)   -  (EP  -  OR) 

or: 

PE  =  ENP  -  EP 

Therefore,  to  express  the  PSRO  effect  as  a  percent  reduction  in 
DOC,  it  is  only  necessary  to  divide  by  the  expected  DOC  (non-PSRO). 
Thus: 


ENP  "  EP 
(8)       PE=  ENP 

An  Alternative  Regression  Specification:  "The  Double  Log  Model": 
It  is  possible  that  for  a  given  time  period  (e.g.,  first  half  of 
1977),  either  the  PSRO  effect  is  independent  of  the  base  value 
(e.g.,  20  DOC  per  1000  eligibles  were  eliminated  through  review 
regardless  of  the  base  rate)  or  that  it  is  proportional  to  the  base 
value  (e.g.,  DOC  is  reduced  two  percent  by  review;  thus,  a  rate 
of  1000  is  reduced  to  980,  a  rate  of  2000  is  reduced  to  1960,  etc.). 
The  constant-effect  (or  additive)  model  suggests  the  direct  use  of 
the  observed  utilization  rates  in  the  statistical  analyses,  while 
the  proportional-effect  (or  multiplicative)  model  suggests  the  use 
of  logarithmic  re-expression  of  the  variables.  To  clarify  this 
distinction,    consider    the    following    idealized   prediction  models: 

a)  Outcome  =  C  +        (Base)  +  (Longev.) 

B  B 

b)  Outcome  =  C  *  Base     1     *  Longev.  2 
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where  Base  and  Outcome  refer  to  the  utilization  rates  in  1974  and 
1977    respectively.  Longev.    is    the   Longevity   variable,    C    is  a 

constant,    and  and   B^    are    regression    coefficients.       Model  a) 

corresponds  to  a  constant-effect  model,  whereas  Model  b)  is  a 
multiplicative  (proportional)  model.  However,  Model  b)  can  be 
transformed  into  an  additive  model  by  employing  logarithms  for  all 
variables : 

b1)  Log  (Outcome)  =  Log  (C)  +  B1  Log  (Base)  +  B2  Log  (Longev.) 

In  many  of  the  analyses,  data  were  entered  in  both  arithmetic  and 
logarithmic  form  in  parallel  series  of  analyses  in  order  to  explore 
the  possible  operation  of  either  additive  or  multiplicative  effects. 

Supplemental  Analyses ;  In  addition  to  the  estimates  derived 
from  the  primary  regression  specification,  the  evaluation  explored: 
(1)  the  variation  of  PSRO  impact  by  region,  (2)  the  relationship  of 
PSRO  impact  to  rate  setting  activities,  (3)  the  degree  to  which  PSRO 
impact  varies  as  a  PSRO  ages  and  (4)  the  relationship  of  estimated 
PSRO  impact  to  days  certified  for  skilled  nursing  facilities  (SNF) 
care. 

(1)  Variation  of  PSRO  impact  by  region.  This  was  estimated  in 
two  ways.  First,  the  results  of  the  disaggregated  analysis  were 
aggregated  (weighting  by  size  of  PSRO)  to  the  regional  level  to 
arrive  at  an  estimate  of  average  PSRO  impact  by  region  of  the 
country.  Second,  region  "dummy"  variables  were  entered  into  the 
regression  equation  to  test  for  regional  effects. 

(2)  PSRO  impact  and  rate  setting  activities.  This  relationship 
was  explored  by  entering  rate  setting  activities  as  a  "dummy" 
variable  in  the  analysis.  That  is,  this  "dummy"  variable  was 
assigned  a  value  of  1  if  the  PSRO  was  located  in  a  state  with  a  rate 
setting  agency,  0  otherwise. 

(3)  PSRO  impact  and  age.  This  was  evaluated  in  two  ways. 
First,  longevity,  as  a  program  variable,  was  included  in  the 
regression  equations.  Second,  the  disaggregated  program  impact 
estimates  were  plotted  by  longevity  in  order  to  check  visually  for 
an  effect  due  to  age. 

(4)  Effect  of  SNF-certif ied  days  on  PSRO  impact.  This  was 
evaluated  by  comparing  estimated  individual  PSRO  impacts  (from  the 
disaggregated  analysis)  with  the  percent  of  all  days  paid  for  by 
Medicare  which  were  certified  at  the  SNF  level  of  care. 
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2.2.4  Findings 


2.2.4.1    Descriptive  Analysis 

Time  Trends  of  Primary  Outcome  Measures;  Figure  2  and  Table  9 
present  the  semi-annual  values  for  primary  outcome  measures  in 
active  (n=96)  and  inactive  (n=93)  PSRO  areas.  The  data  illustrate  a 
number  of  interesting  points.  First,  the  trends  in  both  active  and 
inactive  PSRO  areas  have  followed  similar  patterns  over  time.  DOC 
has  not  changed  appreciably  between  1974  and  1977,  DISC  has  steadily 
risen,  and  ALOS  has  decreased.  Second,  there  is  a  distinct  cyclical 
pattern  in  both  DOC  and  DISC.  In  the  first  calendar  half  of  each 
year,  DOC  are  anywhere  from  40  to  140  days  higher  than  in  the  second 
half.  The  difference  in  DISC  between  the  first  and  second  halves  of 
each  year  ranges  from  two  to  nine  discharges/1,000.  There  are 
several  plausible  hypotheses  for  this  pattern.  It  could  be  that 
during  the  colder  months  in  the  first  half  of  the  year  there  is  more 
sickness.  In  addition,  the  cold  months  in  the  second  half  of  the 
year  (November  and  December)  are  holiday  times  and  discretionary 
hospitalization  may  be  postponed. 

This  observed  pattern  in  utilization  may  affect  PSRO  per- 
formance. Assuming  that  the  available  bed  supply  remains  more  or 
less  constant  throughout  the  year,  occupancy  is  predictably  higher 
in  the  first  half  of  the  year.  In  times  of  high  occupancy,  the 
pressure  is  toward  a  rapid  turnover  of  patients  to  get  new  patients 
in.  In  low  occupancy  times,  there  might  be  a  tendency  to  keep 
patients  hospitalized  longer  and  thus  maintain  revenues.  Thus  PSRO 
activities  might  have  a  better  chance  of  success  in  times  of  high 
occupancy.  Indeed,  the  regression  results  presented  in  2.2.4.2  show 
that  the  PSRO  impact  on  utilization  is  almost  solely  in  the  first 
half  of  the  year.  The  relationship  between  PSRO  effect  and  cyclical 
use  suggests  that  occupancy  may  be  a  controlling  factor. 

A  third  point  is  that  active  and  inactive  areas  have  use  rates 
that  are  quite  similar.  DOC  are  slightly  lower  in  active  areas  but 
only  by  about  one  or  two  percent.  DISC  tend  to  be  five  or  six 
percent  lower  in  active  areas  while  ALOS  is  higher  in  active  areas 
by  about  0.6  days. 

Finally,  the  longitudinal  change  in  the  difference  between 
active  and  inactive  areas  has  not  been  great.  To  the  extent  that 
the  lines  in  Figure  1  remain  more  or  less  parallel,  the  relative 
relationships  between  active  and  inactive  PSRO  have  not  changed 
enough  to  be  visually  apparent. 

Active-Inactive  Comparison:  The  active  and  inactive  PSRO  areas 
were  not  selected  by  a  random  procedure,  but  were  self -selected  in 
the  sense  that  the  decision  of  which  PSROs  to  fund  first  was  a 
process    largely  determined  by   local  initiatives.     Thus,    it  is 
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FIGURE  2:    Medicare  Total  Days  of  Care  per  1,000  Aged  Enrollees, 
Medicare  Discharges  per  1,000  Aged  Enrollees  and 
Average  Length  of  Stay  for  Active  and  Inactive  PSROs 
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possible  that  the  active  and  inactive  PSROs  might  differ  systema- 
tically with  respect  to  base  utilization  rates  and/or  adjustment 
variables.  To  check  for  this  possibility,  a  multivariate  comparison 
was  made  between  the  96  active  and  the  93  inactive  PSROs  using  the 
Hotelling  T  statistic./ 1  The  variables  entering  into  the  compar- 
ison were:  occupancy  rate,  poverty  level,  density,  percent  DOC  due 
to  Medicare  enrollees,  percent  DISC  due  to  Medicare  enrollees, 
nursing  home  beds/1, 000  (1973),  physicians/ 1 ,000  (1974),  percent 
aged  (1974),  short-stay  beds/1, 000  (1974),  total  DOC  (1974),  live 
DOC  (1974),  total  DISC  (1974),  live  DISC  (1974),  total  ALOS  (1974), 
live  ALOS  (  1974),  percent  short  . stays  (1974)  and  percent  long  stays 
(1974). 

Overall,  there  is  no  evidence  for  non-comparability  of  the 
active  and  inactive  sites.  The  value  of  Hotelling' s  T  statistic 
is  29.00  which,  with  17  and  171  degrees  of  freedom,  is  non-signifi- 
cant at  conventional  levels  (F=1.560,  p=.080). 

Although  the  overall  comparison  of  means  was  non-significant, 
the  two  groups  are  by  no  means  identical.  Certain  of  the  variables 
did  show  significant  mean  differences  when  viewed  individually. 
Thus,  the  active  PSROs  were  higher  on  the  average  than  the  inactive 
PSROs  with  respect  to  number  of  MDs  per  1000  population  (1.66 
versus  1.25),  population  density  (2700  per  square  mile  versus  515), 
and  percent  of  long  hospital  stays  (13.09  versus  11.74),  while  the 
inactive  PSROs  were  higher  on  the  average  than  the  active  PSROs  with 
respect  to  total  discharge  rate  (328.23  versus  311.82)  and  live 
discharge  rate  (302.91  versus  287.19). 

Table  10  compares  the  means  of  active  and  inactive  PSROs  on 
these  variables  as  well  as  1977  utilization  rates  and  the  1977 
adjustment  variables  that  were  available.  From  Table  10,  one  can 
calculate  the  gross  change  in  total  DOC  for  both  active  and  inactive 
PSROs.  Active  PSROs  experienced  a  three-tenths  of  a  percent 
increase  in  total  DOC  from  1974  to  1977  [  ( 3678-3668)/3668]  while 
inactive  PSROs  experienced  an  eight-tenths  of  a  percent  increase  in 
total  DOC  from  1974  to  1977  [ ( 3727-3699)/3699] .  This  is,  of  course, 
unadjusted  for  demographic  differences.  The  regression  analyses 
that  follow  will  control  for  these  exogenous  variables,  resulting  in 
an  estimated  program  effect  which  will  differ  from  the  0.5  percent 
difference  presented  above. 


/1  The  T     statistic  is  the  multivariate  extension  of  the  ordinary 
Student  t-test  for  the  difference  bewen  two  sample  means. 
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TABLE  10 


UTILIZATION  RATES  FOR  MEDICARE  ENROLLEES  (1974,  1977) 

and 

ANNUAL  ADJUSTMENT  VARIABLE  VALUES  BY  ACTIVE  AND 
INACTIVE  PSRO  GROUPS 

Active  Inactive 
All  PSROs      PSROs  PSROs 
(N=189)       (N-96)  (N=93) 

Characteristic 


1974  Utilization  Rates 


Total  Days  of  Care/ 1,000 

3683 

3668 

3699 

Live  Days  of  Care/ 1,000 

3368 

3352 

3384 

Total  Discharges/ 1 ,000 

320 

312* 

328* 

Live  Discharges/ 1 ,000 

295 

287* 

303* 

Avg.  Length  of  Stay  (Total) 

11.7 

11.9 

11.4 

Avg.  Length  of  Stay  (Live) 

11.6 

11.9 

11.3 

Percent  Short  Stays  (1-3  days) 

17.0% 

17.0% 

17.1% 

Percent  Long  Stay  (22+  days) 

12.4% 

13.1%* 

11.6%* 

1977  Utilization  Rates 


Total  Days  of  Care/ 1,000 

3703 

3678 

3727 

Live  Days  of  Care/ 1,000 

3384 

3354 

3414 

Total  Discharges/ 1 ,000 

342 

333 

351 

Live  Discharges/ 1 ,000 

316 

307 

325 

Avg.  Length  of  Stay  (Total) 

10.9 

11.2 

10.7 

Avg.  Length  of  Stay  (Live) 

10.8 

11.0 

10.6 

Percent  Short  Stays  (1-3  days) 

19.1% 

19.1% 

19.1% 

Percent  Long  Stay  (22+  days) 

10.8% 

11.4% 

10.3% 

Adjustment  Variables 

Poverty  Fami lies/ 1 ,000  persons 

110 

109 

111 

Population/Square  Mile 

1625 

2700* 

515* 

Percent  of  all  Days  of  Care 

Due  to  Medicare  Patients 

33.4% 

31.1% 

33.6% 

Percent  of  all  Discharges  Due 

to  Medicare  Patients 

22.8% 

22.4% 

23.1% 

Occupancy  Rate 

74% 

74% 

73% 

Nursing  Home  Beds/1,000  (1973) 

5.9 

6.0 

6.4 

Nursing  Home  Beds/1,000  (1976) 

6.3 

6.2 

6.4 

Short  Stay  Beds/1,000  (1974) 

4.3 

4.3 

4.3 

Short  Stay  Beds/1,000  (1976) 

4.4 

4.3 

4.4 

Percent  Aged  (1974) 

10.2% 

10.1% 

10.3% 

Percent  Aged  (1976) 

10.5% 

10.4% 

10.7% 

Physicians/ 1,000  (1974) 

1.5 

1.7%* 

1.2%* 

Physicians/ 1,000  (1976) 

1.6 

1.8 

1.3 

Denotes  characteristics  for  which  there  was  a  statistically 
significant  difference  (P<.05)  between  active  and  inactive 
PSRO  areas  in  the  base  year. 
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2.2.4.2    Regression  Results;     Days  of  Care 


This  section  presents  findings  related  to  the  outcome  variable 
"total  days  of  care".  The  estimate  of  PSRO  impact  derived  from  the 
primary  regression  specification  model  is  presented.  This  model  is 
based  on  annual  data  and  estimates  PSRO  effect  at  the  national 
level.  Disaggregated  results  are  then  presented.  To  supplement  the 
aggregate  and  disaggregate  findings/  the  results  of  numerous  alter- 
native regression  runs  based  on  semiannual  data,  various  measures  of 
the  program  impact  variable,  and  the  double  log  regression  model  are 
presented.  The  section  concludes  with  results  from  supplemental 
analyses  related  to  DOC. 

The  Primary  Regression  Specification  Estimated  with  Annual 
Data;  Table  11  presents  the  aggregate  regression  results  for  total 
DOC  using  annual  data.  The  PSRO  program  variable,  longevity, 
represents  the  number  of  months  a  PSRO  had  been  in  operation  as  of 
April  1,  1977. 

TABLE  11 

TOTAL  DOC  SUMMARY  OF  PROPORTIONAL  INCREMENTS  TO  EXPLAINED  VARIANCE 
ANNUAL  DATA:     ANALYSIS  IN  RAW  UNITS 

Independent                               Increase       Cumulative  Significance 
Variable  Stage/ 1  in  R   R    Level 

Total  Days  of 
Care/ 1 ,000 

(1974)  1  .860  .860  .001 

Adjustment 

Variables  2  .069  .929  .001 

(a)  Short-Stay 
Beds/1, 000(+) 

(b)  Population 
Density (+) 

( c )  Occupancy 
Rate(+) 

PSRO  Review 
Variable 

(Longevity)  3  .002  .931  .05 

Interaction  Terms 
Base  rate  vs. 

longevity  4  .000  .931  NS 

Adjustment 

variables  vs. 

longevity  4  .004  .935  NS 

+  denotes  the  sign  of  the  regression  coefficient  of  the  adjustment 
variables  which  were  statistically  significant.  NS  denotes  "not 
significant". 


/t  Order  in  which  variables  are  entered  into  the  regression. 
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The  overall  explained  variance  (R  )  for  this  equation  is 
.935.  Most  of  the  explained  variance  is  due  to  the  inclusion  of  the 
base  rate.  That  is,  the  best  predictor  of  1977  DOC  is  1974  DOC. 
The  adjustment  variables  in  sum  account  for  another  6.9  percent  of 
the  total  variance.  The  significant  adjustment  predictors  of 
DOC  were  short-stay  beds/1 ,000  population,  population  density  and 
occupancy  rate.  That  is,  increases  in  beds,  population  density  and 
occupancy  tend  to  be  associated  with  increases  in  DOC.  Both  the 
base  rate  and  the  sum  of  the  adjustment  variable  effects  were 
significant  at  the  .001  level. 

PSRO  longevity  is  seen  to  account  for  two-tenths  of  one  percent 
of  the  total  variance.  Although  small,  this  additional  contribution 
was  significant  at  the  .05  level. 

Interactions  of  PSRO  longevity  with  the  base  rate  and  the 
adjustment  variables  were  also  tested.  Neither  the  interaction  of 
the  base  rate  with  longevity  nor  the  sum  of  the  other  interaction 
terms  significantly  increased  the  explained  variance. 

The  results  of  this  part  of  the  regression  analysis  support 
the  appropriateness  of  a  disaggregated  analysis  of  individual 
PSROs.  Had  the  program  effect  not  been  significant,  such  disaggre- 
gated analysis  would  be  equivalent  to  examination  of  random  varia- 
tion of  points  around  a  zero  effect,  a  fairly  meaningless  analysis. 
However,  since  the  program  effect  (longevity,  in  this  instance)  is 
statistically  significant,  the  analysis  of  individual  PSRO  effects 
is  appropriate. 

The  regression  results  thus  far  indicate  only  that  the  PSRO 
impact  is  significant.  The  magnitude  and  direction  of  that  impact 
is  noted  by  an  inspection  of  the  regression  coefficients.  Table  12 
contains  two  sets  of  unstandardized  regression  coefficients. /1  The 
first  set  of  coefficients  relates  to  the  model  before  the  inter- 
action terms  were  added  while  the  second  set  of  coefficients  relates 
to  the  full  regression  model.  In  the  regression  model  without 
interaction  terms ,  all  of  the  program  impact  is  included  in  the 
longevity  coefficient  and  is  not  spread  among  many  interaction 
terms.  As  can  be  seen,  the  coefficient  is  in  the  expected  negative 
direction  (-2.8640). 


/ 1  The  unstandardized  regression  coefficient  represents  the  ex- 
pected change  in  the  dependent  variable  for  each  unit  change  in 
the  independent  variable. 
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TABLE  12 


RAW  REGRESSION  COEFFICIENTS  FOR  PREDICTION  MODEL 
OF  TOTAL  DAYS  OF  CARE:     PSRO  LONGEVITY  AS  A  PREDICTOR 


Independent 
Variables 

Base 

Occupancy 
Poverty 
Density 
Short- Stay 

Beds/1,000 
Nursing  Home 

Beds/1,000 
MDs/1,000 
%  Aged 
%  DOC  Due  to 

Medicare 
PSRO  Longevity 
Base  X  Longevity 
Occupancy 

X  Longevity 
Poverty 

X  Longevity 
Density 

X  Longevity 
Short -Stay  Beds 

X  Longevity 
Nursing  Beds 

X  Longevity 
MDs  X  Longevity 
%  Aged 

X  Longevity 
%  DOC 

X  Longevity 
Constant 


Coefficients : 
Model 
Without 
Interactions 

.8303 
2308.6144 
-.0276 
.0069 

415.7746 

-19.7548 
.0486 
-10.6667 

.  1255 
-2.8640 


Partial 
(F  value) 

( 1016.8) 
(87.7) 
(.0) 
(8.9) 

(39.5) 

(3.1) 
(0.0) 
(3.7) 

(.3) 
(4.7) 


-1069.7936 


Coefficients ; 

Model  With 
Interactions 

.8146 
2306.5160 
-.5438 
.0118 

409.8322 

-23.4140 
74.9610 
-18.7645 

.4203 
3.1587 
.0021 

-12.7120 

.0718 

-.0004 

1.3524 

.7636 
-5.3079 

1 .3435 

-.0466 
-1030.0649 


Partial 
(F  value)/1 

(503.5) 
(49.9) 
(0.7) 
(2.7) 

(24.3) 

(3.0) 
(0.1) 
(6.5) 

(1.8) 
(0.0) 
(.7) 

(.3) 
(1.8) 

(.6) 

(0.0) 

(0.4) 
(0.1) 

(4.7) 

(2.2) 


Days  of  Care  Per  Thousand: 

Active  plus  inactive  PSROs 

Inactive  only 

Active  only 

Percent  Difference 


3702.5 
3732.4/2 
3663.5/2 
1 .58% 


/ 1  F  values  greater  than  4.0  are  significant  at  the  .05  level. 

/2  Predicted  based  on  average  overall  values  for  base  and  adjustment 
variables . 
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By  multiplying  the  program  impact  variable  coefficient  by  the 
average  value  of  PSRO  longevity  (15.5  months)/  an  estimated  -44.39 
days  is  obtained  for  PSRO  impact  (-2.8640  X  15.5  =  -44.39).  This 
estimate,  however,  does  not  take  into  account  the  information  of 
the  full  model  in  that  the  interaction  terms  are  omitted. 

When  interpreting  the  coefficients  of  the  full  model  it  should 
be  noted  that  the  presence  of  the  interaction  terms  affects  the 
remaining  coefficients.  Thus,  in  Table  12  the  PSRO  Longevity 
coefficient  for  the  full  model  is  3.1587.  This  should  not  be 
interpreted  to  mean  that  the  PSRO  effect  tends  to  increase  days  of 
care.  Rather,  the  average  PSRO  effect  involves  the  product  terms  as 
well,  i.e.,  Base  times  Longevity  through  %  DOC  due  to  Medicare  times 
Longevity,    several   of    which    show   strong,    negative  contributions. 

The  interpretation  of  the  full  model,  then,  involves  an 
analysis  of  the  PSRO  longevity  variable  and  the  interaction  terms. 
By  way  of  illustration,  the  lower  section  of  Table  12  compares  two 
hypothetical  PSROs.  Both  are  assumed  to  be  completely  average  with 
respect  to  the  base  and  adjustment  variables.  One  PSRO  is  assumed 
active  with  15.5  months  of  review;  the  other  inactive. 

As  shown  in  Table  12  for  Total  DOC,  the  overall  mean  is  3702.5 
days  of  care/1000  eligibles  on  an  annual  basis.  However,  for 
the  completely  average  inactive  PSRO,  the  predicted  average  DOC 
rate  would  be  3732.4  days  of  care/1000  eligibles,  but  only  3663.5 
days  of  care/1000  eligibles  for  a  comparable  active  PSRO.  The 
difference  is  1.58  percent  which  represents  the  average  savings  in 
DOC  due  to  the  PSRO  review  procedure. 

The  Primary  Regression  Specification  -  Disaggregate;  Estimates 
based  on  annual  data:  The  full  equation  as  specified  in  Table  12 
was  used  in  the  disaggregated  analysis.  It  should  be  noted  that 
individual  PSRO  impacts  will  not  be  as  accurate  as  the  estimate  of 
overall  effect.  This  is  because  the  accuracy  of  individual  predic- 
tions is  a  function  of  two  things:  the  overall  fit  of  the  model  and 
the  degree  to  which  any  individual  PSRO  may  be  an  outlyer  from  the 
general  distribution.  The  overall  fit  of  the  model  is  quite  good 
(R  =  .94).  However,  individual  error  will  still  occur  and  the 
results  of  the  disaggregated  analysis  should  be  interpreted  with 
some  caution. 

Table  13  presents  the  individual  PSRO  effects  aggregated  by 
region  and  by  density.  Note  that  the  overall  average  decrease  is 
1.5%.  This  differs  slightly  from  the  1.58%  given  in  Table  12 
because  the  figure  in  Table  12  is  an  overall  estimate  while  the 
figure  in  Table  13  is  a  weighted  average  of  individual  PSRO  effects 
(the  weighting  was  on  the  basis  of  Medicare  enrollment). 
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TABLE  13 

PERCENT  DECREASE  IN  TOTAL  DAYS  OF  CARE  PER  1,000  MEDICARE 
ENROLLEES  DUE  TO  PSRO  REVIEW,  BY  REGION  AND  DENSITY 


PSRO  Grouping 

All  active  PSROs  (n=96) 
Region  ( census ) 

Northeast  (n=32) 

North  Central  (n=21) 

South  (n=19) 

West  (n=24) 
Region  (HEW) 

1  (n=9) 

2  (n=15) 

3  (n=20) 

4  (n=6) 

5  (n=13) 

6  (n=1) 

7  (n=6) 

8  (n=6) 

9  (n=14) 
10  (n=5) 

Density  (Persons  per  square  mile) 
10,000+  (n=8) 
1,000-9,999  (n=15) 
500  -  999  (n=10) 
100  -  499  (n=23) 
Less  than  100  (n=40) 


Percent  Decrease  in 

-  1.5% 

-  2.7% 

-  .5% 
.0% 

-  2.6% 

-  3.2% 

-  3.2% 

-  1.2% 
+  .6% 

.4% 

-  1.1% 

-  .3% 

-  1.7% 

-  2.2% 

-  3.1% 

-  3.5% 

-  1.5% 

-  2.4% 

-  1.2% 

-  .8% 
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The  impact  of  PSRO  on  DOC  is  not  constant  by  region  of  the 
country.  PSROs  in  the  Northeast  and  West  have  the  greatest  impact 
(-2.7%  and  -2.6%  respectively)  while  the  North  Central  PSROs  have 
only  a  small  impact  (-0.5%)  and  PSROs  in  the  South  have,  on  average, 
no  impact.  A  finer  regional  breakdown  into  the  ten  HEW  regions 
shows  that  PSROs  in  regions  1,  2  and  10  have  the  greatest  impact 
with  reductions  of  3.2%,  3.2%  and  3.1%  respectively.  PSROs  in 
regions  8  and  9  reduced  DOC  by  1.7%  and  2.2%  respectively.  Other 
regions  showed  decreases  of  only  one  percent  or  less  and  PSROs  in 
region  4  actually  showed  an  increase  in  DOC  of  .6%. 

Table  13  also  indicates  that  urban  PSROs  (as  measured  by 
population  density)  are  more  effective  than  less  urban  PSROs.  PSROs 
with  population  density  of  10,000  or  more  per  square  mile  reduced 
DOC  by  3.5%,  while  the  smallest  decreases  (.8%)  were  shown  by  the 
least  urban  (lowest  density)  PSROs  (less  than  100  persons  per  square 
mile).  This  finding  has  important  implications  for  the  future 
success  of  the  PSRO  program.  Most  of  the  urban  PSROs  were  already 
operational  as  of  1977.  The  remaining  inactive  PSROs  are  largely 
rural  (less  dense)  in  nature.  If  the  relationship  between  density 
and  impact  continues,  one  would  expect  the  overall  program  impact  to 
decrease  as  the  newer  PSROs  become  operative. 

Table  14  presents  an  estimate  of  PSRO  impact  on  DOC  for  each 
of  the  96  active  PSROs  included  in  the  study.  The  PSROs  are  ranked 
according  to  percent  reduction  in  DOC  due  to  PSRO  review.  All  but 
15   of   the  PSROs   showed  some  reduction  in  DOC  due  to  PSRO  review. 

It  is  important  to  note  again  that  the  percent  reduction  in  DOC 
estimated  for  each  active  PSRO  does  not  equate  with  the  observed 
reduction  in  DOC.  Other  non-observed  factors  have  an  impact  on  DOC. 
The  effect  of  those  non-observed  factors  is  represented  by  the  error 
term.  Thus,  2NY10  (Rank  #10)  has  a  5.34%  decrease  in  DOC  due  to 
PSRO  review.  However,  a  7.11%  increase  in  DOC  due  to  unknown 
factors  resulted  in  a  slight  (1.77%)  overall  increase  in  DOC, 
despite  the  decrease  attributable  to  PSRO  review. 

The  total  number  of  days  saved  (as  opposed  to  a  percent  reduc- 
tion) is  a  function  both  of  the  percent  reduction  in  DOC  and  the 
size  of  the  PSRO.  The  total  days  saved  calculation  was  done  in 
order  to  perform  the  benefit-cost  analysis  of  Section  4  and  can  be 
found  in  that  Section. 

As  was  mentioned  earlier,  the  individual  PSRO  effect  estimates 
are  subject  to  a  greater  degree  of  error  than  the  overall  estimate. 
As  the  individual  PSROs  are  aggregated  into  larger  groupings  (e.g., 
regional  groupings),  the  errors  tend  to  "even  out"  so  that  groups  of 
PSROs  are  more  meaningful  to  examine. 

Thus,  the  ranks  of  individual  PSROs  should  be  considered  as 
rough  order  of  magnitude  estimates  rather  than  precise  point  esti- 
mates. 
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TABLE  14 

PERCENT  REDUCTION  IN  DOC  DUE  TO  PSRO  REVIEW 
BY  PSRO 


%    Ro  /%ipf  4  rxr\ 

PSRO 

in  DOC  due 

Total 

dviG  to 

Desiomation/ 1 

Rank 

to  PSRO  = 

T?ea             -5  ryn 

Rpvi pw / 2 

1»C  V  J-  ^  W  / 

in  DOC 

8WY0  1 

1 

-8.75 

-4.42 

4.33 

9CA12 

2 

-7.49 

-3.86 

3.63 

2NY1 1 

3 

-6.53 

-10.43 

-3.90 

2NY16 

4 

-6.42 

-1 .96 

4.46 

9CA0 1 

5 

-6. 10 

2.07 

8. 17 

9CA27 

6 

-5.71 

-9.92 

-4.21 

2NY13 

7 

-5.66 

-9.31 

-3.65 

1CT0 1 

g 

-5.51 

-3 .60 

1.91 

■  •71 

3MD03 

9 

-5.37 

-9.60 

-4.23 

2NY10 

10 

-5. 34 

1.77 

7.11 

10OR0 1 

1 1 

-5.10 

-1 1 . 36 

-6.26 

9CA03 

12 

-5.00 

-1.84 

3.16 

8UT0 1 

13 

-4.63 

-1 1 .47 

-5.84 

5IL04 

13 

-4.63 

-7.53 

-2.90 

3PA1 1 

15 

-4.56 

-3.67 

.89 

10WA0 1 

16 

-4. 34 

-4.49 

-.15 

1MA0 1-05 

17 

-4. 12 

2.93 

7.05 

2NY02 

17 

-4. 12 

1.82 

5.94 

3MD05 

19 

-4.06 

-7.67 

-3.61 

1CT04 

20 

-3.64 

-11 .26 

-7 .62 

2NY09 

21 

-3.52 

-.51 

3.01 

1ME0  1 

22 

-3.49 

-.46 

3.03 

3MD07 

23 

-2.63 

1 .99 

4.62 

6NM0 1 

24 

-2.61 

-2 .96 

-.35 

1NH0 1 

25 

-2.59 

-4.44 

-1 .85 

1 0 IDO 1 

26 

-2.58 

-6.77 

-4.19 

9CA18-26 

27 

-2.32 

-1 .78 

.54 

1CT03 

28 

-1 .93 

-8.91 

-6.98 

1CT02 

29 

-1 .88 

-6.71 

-4.83 

9CA05 

30 

-1 .77 

4.17 

5.94 

5WI02 

30 

-1.77 

-6.96 

-5. 19 

3PA09 

32 

-1 .56 

-3.23 

-1 .67 

3MD0 1 

33 

-1 .48 

5.10 

6.58 

3PA1 2 

34 

-1 .46 

6.13 

7.59 

Ovtv  1  A 
X  I  ft 

jj 

—  1  AA 

D  .  /  3 

D  .  3  1 

3DC01 

36 

-1.39 

-2.43 

-1.04 

9CA08 

36 

-1.39 

2.69 

4.08 

3PA07 

38 

-1.36 

-.60 

.76 

5MI01 

39 

-1.27 

-3.19 

-1.92 

/1  PSRO  designations  follow  the  format  used  by  HSQB.  The  first 
number  is  the  HEW  region;  the  letters  are  the  state  abbre- 
viation; the  last  two  numbers  identify  the  PSRO  number  within 
the  state.  Thus,  3MD04  denotes  PSRO  Number  04  in  Maryland/ 
which  is  in  HEW  region  3. 

/2  Percent  reduction  in  DOC  due  to  PSRO  review  was  calculated 
using  the  formulation  (8)  shown  on  page  62,  where  ENP  and  EP 
refer  to  the  expected  non-PSRO  and  PSRO  days  of  care  respec- 
tively, as  derived  through  regression  using  area-specific  demo- 
graphic and  adjustment  values. 


TABLE  14  -  Continued 


%  Reduction  %  Reduction 

PSRO  in  DOC  due  Total  due  to 

Designation         Rank         to  PSRO  -      Reduction  -  non-observed 

Review  in  DOC       factor  (error) 


7MO05 

40 

-1.24 

2.93 

4.17 

4FL12 

41 

-1.23 

-.25 

.98 

9CA04 

42 

-1.17 

-1.66 

-  .49 

10OR02 

43 

-1.14 

-13.40 

-12.26 

2NJ01 

44 

-1.12 

1.33 

2.45 

5MN0  2 

45 

-1.02 

-6.21 

-5.19 

7MO02 

46 

-1.01 

-.98 

.03 

9CA17 

47 

-.97 

4.34 

5.31 

8MT0  1 

48 

-.94 

-3.04 

-2.10 

2NY0  1 

49 

-.93 

2.81 

3.74 

9HI01 

50 

-.92 

-5.16 

-4.24 

8C001 

51 

-.87 

1.19 

2.06 

6AR01 

52 

-.66 

-2.94 

-2.28 

3DE01 

53 

-.65 

-.68 

-.03 

7IA01 

54 

-.58 

-2.47 

-1.89 

5  OHO  4 

55 

-.57 

-2.90 

-2.33 

5IL03 

56 

-.56 

-.37 

.  19 

5OH10 

57 

-.44 

1 . 18 

1.62 

8SD0 1 

58 

-.43 

.60 

1.03 

2NJ02 

58 

-.43 

-1.49 

-1.06 

2NY15 

60 

-.37 

5  .22 

5  .59 

3VA05 

61 

-.34 

-8.97 

-8.63 

3WV0 1 

62 

-.32 

-4.79 

-4.47 

5MN03 

63 

-.31 

-5  .94 

-5.63 

5OH01 

63 

-.31 

-2.60 

-2.29 

2NJ04 

63 

-.31 

1 .88 

2. 19 

9CA09 

66 

-.29 

-7.04 

-6.75 

9CA14 

67 

-.28 

.23 

.51 

4KY0  1 

68 

-.27 

1.86 

2.13 

1VT01 

69 

-.26 

3.30 

3.56 

3MD04 

70 

-.22 

14.55 

14.77 

5  OHO  2 

70 

-.22 

3.01 

2.79 

2NY12 

70 

-.22 

-.08 

.14 

1RI01 

73 

-.19 

-8.74 

-8.55 

5WI01 

74 

-.10 

-5.59 

-5.49 

3PA04 

74 

-.10 

-.30 

-.20 

7MO03 

74 

-.10 

.60 

.70 

2NY04 

74 

-.10 

2.34 

2.44 

3VA02 

78 

-.06 

-5  .87 

-5.81 

5MI05 

79 

-.05 

-.83 

-.78 

3MD06 

80 

-.03 

-4.26 

-4.23 

9NV01 

81 

-.01 

.62 

.63 

8ND01 

82 

+  .04 

-.14 

-.18 

2NY05 

83 

+  .14 

.52 

.66 

7MO01 

84 

+  .15 

2.68 

2.53 

4AL01 

85 

+  .22 

2.21 

1.99 

7MO04 

86 

+  .26 

-1.07 

-1.33 

3PA08 

87 

+  .30 

2.04 

1.74 
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TABLE  14  -  Continued 


%  Reduction 

%  Reduction 

PSRO 

in  DOC  due 

Total 

due  to 

Designation 

Rank 

to  PSRO 

Reduction 

-  non-observed 

Review 

in  DOC 

factor  (error) 

50H12 

88 

+  .36 

-4.20 

-4.56 

3PA06 

89 

+  .43 

-1.65 

-2.08 

10AK01 

90 

+  .53 

5.78 

5.25 

3MD02 

91 

+  .55 

9.89 

9.34 

3PA02 

92 

+  .82 

-3.22 

-4.04 

9CA16 

93 

+  1.01 

-1.10 

-2.11 

4MS01 

94 

+  1.15 

6.47 

5.32 

4TN02 

95 

+  1.77 

1.20 

-.57 

4SC01 

96 

+  1.96 

7.24 

5.28 

Alternative  Regression  Specifications;  Several  additional 

regression  models  were  run  on  DOC  because  of  different  hypotheses 
as  to  how  the  PSRO  program  effect  might  be  operating.  PSRO  effect 
may  be  a  function  of  longevity  (length  of  time  PSRO  has  been  in 
operation)  or  intensity  (percent  of  all  Medicare  discharges  under 
review).  The  relative  effect  of  these  variables  needed  to  be 
testes;  regressions  were  run  to  test  for  those  effects.  PSRO  effect 
may  be  an  absolute  effect  or  it  may  be  proportional  to  the  pre-PSRO 
use  rate  (i.e.,  PSROs  may  have  less  absolute  effect  in  low  use 
areas).  Regressions  were  run  on  total  days  of  care  and  live  days  of 
care  for  both  the  first  and  second  halves  of  1977. '  This  resulted  in 
a  multitude  of  equations  which,  had  the  results  been  greatly 
different,  would  have  been  difficult  to  interpret.  However,  the 
regression  equations  under  all  the  different  model  specifications 
produced  essentially  the  same  results.  The  regression  results  which 
follow  are  presented  to  place  the  evaluation's  primary  regression 
analysis  in  perspective. 

(a)  Semi -Annual  Regressions  -  Data  in  Raw  Units;  Table  15 
presents  the  regression  rund  where  the  PSRO  variable  is  expressed  in 
terms  of  longevity  and  intensity.  The  outcome  variables  are  live 
days  of  care,  first  and  second  half  of  the  year  (L  DOC  F  and  L  DOC 
S)  and  total  days  of  care,  first  and  second  half  of  the  year  (T  DOC 
F  and  T  DOC  S) .    The  table  summarizes  eight  regression  runs. 

2 

The  overall  explained  variance  (R  )  for  these  equations 
ranges  from  .91  to  .93.  Most  of  the  explained  variance  is  due 
to  the  inclusion  of  the  base  rates.  That  is,  the  best  predictor 
of  days  of  care  for  the  first  half  of  1977  is  the  days  of  care  for 
the  first  half  of  1974.  The  adjustment  variables  in  sum  account 
for  another  five  to  eight  percent  of  the  explained  variance. 
The  significant  adjustment  variables  were  short-stay  beds/1, 000 
population,  population  density  and  occupancy  rate.  That  is,  in- 
creases in  these  factors  tended  to  be  associated  with  increases  in 
days  of  care.  Both  the  base  rate  and  the  sum  of  the  adjustment 
effects  were  significant  at  the  .001  level  of  probability. 
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When  measured  as  longevity,  the  program  impact  variable  ac- 
counted for  one-half  of  one  percent  of  the  explained  variance  (for 
the  first  half  of  1977  only).  This  was  significant  at  the  .005 
level  for  both  total  and  live  days  of  care.  When  impact  was  mea- 
sured as  intensity,  the  explained  variance  was  slightly  lower 
(four-tenths  of  one  percent)  but  was  still  significant  (.01  for  live 
days  of  care  and  .05  for  total  days  of  care).  In  no  case  was  there 
a  significant  program  effect  on  days  of  care  for  the  second  half  of 
1977.  Delegation  status  was  entered  in  the  regression  for  intensity 
and  contributed  nothing  to  the  explained  variance.  Interactions  of 
PSRO  review  with  the  base  rate  and  the  adjustment  variables  did  not 
significantly  increase  explained  variance. 

The  nursing  home  beds/program  interaction  was  significant 
but  only  for  LDOC  in  the  second  half  of  the  year.  This  interaction 
effect  tentatively  suggests  that  PSROs  in  areas  with  greater  numbers 
of  nursing  home  beds  may  be  slightly  more  effective. 

Regressions  were  also  run  to  test  for  the  incremental  effects 
of  longevity  of  review  and  intensity  of  review.  These  regressions 
are  summarized  in  Table  16. 

Regressions  were  run  for  the  first  half  of  the  year  only, 
since  no  PSRO  effect  had  been  found  for  the  second  half.  The 
regressions  are  the  same  as  in  Table  15  except  that  the  specifica- 
tion of  the  program  variable  was  changed.  First,  the  program 
variable  is  entered  in  dummy  form  (0/1),  that  is,  an  area  is  given  a 
value  of  zero  if  inactive  and  one  if  active,  no  matter  what  the 
level  of  activity.  Next,  a  continuous  measure  of  level  of  activity, 
either  longevity  or  intensity,  is  entered  to  test  for  incremental 
effects. 

Table  16  shows  that  no  significant  increment  in  explained 
variance  results  from  treating  the  program  variable  as  continuous. 
The  PSRO  impact  on  days  of  care  is  apparently  an  initial  effect 
which  increases  neither  with  time  nor  with  intensity.  One  charac- 
teristic of  the  PSRO  program  may  help  to  explain  this  apparently 
anomalous  finding.  Once  PSROs  start  review,  they  generally  reach  a 
very  high  level  of  review  (90%  or  more  of  the  discharges)  very 
quickly.  Consequently  there  is  little  variation  among  active  PSROs 
in  level  of  review;  most  have  a  level  of  review  of  90%  or  better. 
Because  these  high  levels  are  reached  early,  this  also  affects  the 
longevity  variable.  If  a  100%  implementation  status  is  reached  in 
the  first  year,  the  PSRO  has  as  much  control  over  use  as  it  is  going 
to  have.  An  additional  one,  two  or  three  years  of  experience  may 
improve  its  operating  capabilities  but  not  its  coverage.  Thus,  if 
it  is  effective  in  the  first  year  it  will  be  difficult  to  improve 
that  effectiveness  with  time.  The  effect  of  longevity  is  explored 
further  in  Section  2.2.4.3  where  some  tentative  evidence  is  pre- 
sented to  support  a  slight  tendency  towards  improved  PSRO  impact  as 
PSROs  mature. 
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Double  Log  Estimation  Equation:  The  previous  regressions 
tested  for  a  PSRO  effect  that  is  constant  with  respect  to  the  days 
of  care  rate.  That  is,  the  effect  was  assumed  to  be  independent  of 
the  actual  days  of  care.  However,  it  is  also  possible  that  the 
effect  is  proportional  to  the  actual  days  of  care  rate.  Logarithmic 
transformation  of  the  data  allows  this  hypothesis  to  be  tested. 
Table  17  presents  the  results  of  the  regressions  in  double  log 
form. 

The  results  in  logarithmic  units  generally  parallel  those  in 
raw  units  (Table  15).  The  total  explained  variance  is  higher  in 
log  units  (.92  to  .95)  as  is  the  explained  variance  due  to  PSRO 
review  (by  two-tenths  or  three-tenths  of  a  percent).  The  effect 
of  PSRO  review  is  small  but  significant.  As  with  the  previous 
regressions,  the  effect  is  found  only  for  the  first  half  of  the 
year.  The  results  of  the  regressions  in  log  units  suggest  that  the 
PSRO  effect  may  indeed  be  proportional  to  the  overall  days  of  care 
rate. 


2.2.4.3    Supplemental  Analyses 

Regional  Analysis;  As  shown  in  the  disaggregated  analysis 
in  Section  2.2.4.2,  there  was  considerable  variation  in  PSRO  effec- 
tiveness by  region  of  the  country.  Therefore,  a  regression  equation 
was  modeled  with  the  addition  of  "dummy"  variables  to  represent  the 
four  regions  of  the  country.  The  results  of  this  analysis  are 
presented  in  Table  18. 

Several  conclusions  can  be  drawn  from  this  regression.  First, 
there  are  significant  regional  differences  in  total  DOC.  The 
Northeast  is  about  200  DOC  higher  than  the  South  which  is  second 
highest.  The  North  Central  is  next  and  the  West  has  a  rate  about 
300  DOC  lower  than  the  Northeast,  after  controlling  for  differences 
in  the  adjustment  variables.  Second,  the  PSRO  effect  remains  after 
controlling  for  regional  variation;  in  fact,  the  level  of  signifi- 
cance is  greater  than  when  regional  dummy  variables  are  omitted. 
Third,  there  is  a  significant  interaction  (at  the  .05  level)  between 
the  PSRO  program  variable  and  region  of  the  country.  Fourth,  the 
estimates  of  percent  reduction  in  total  DOC  due  to  PSRO  review  for 
the  Northeast,  North  Central,  South,  and  West  regions  are  -4.0%, 
-1.6%,  +1.3%  and  -2.0%,  respectively.  Although  the  ordering  remains 
the  same,  the  magnitude  of  these  effects  differs  from  that  of  the 
regional  analysis  of  Section  2.2.4.2.  It  should  be  noted,  however, 
that  the  comparisons  being  made  differ  in  the  two  analyses.  In  the 
primary  analysis  (without  regional  dummies),  active  PSROs  are 
compared  with  inactive  PSROs  nationally.  When  the  dummy  variables 
are  entered,  the  comparison  essentially  becomes  one  of  active/ 
inactive  within  region.  Thus,  when  Northeast  PSROs  are  evaluated 
against   national  trends,    their   relative   reduction  is   2.7%.  When 
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TABLE  18 


PROPORTIONAL  INCREMENTS  TO  EXPLAINED  VARIANCE 
AND  ESTIMATED  PSRO  EFFECT  ON  TOTAL  ANNUAL  DOC: 

BY  REGION 


Independent 
Variable 

Stage 

Increase 
in  R 

Cumulative 
R 

Significance 
Level 

Base  rate 

1 

.860 

.860 

.001 

Adjustment 
Variables 

2 

.069 

.929 

.001 

Regional  Dummy 
Variables 

3 

.007 

.936 

.005 

PSRO  Longevity 

4 

.002 

.938 

.01 

Interactions : 

Base  X  Longevity 

5 

.000 

.938 

NS 

Base  X  Adjustment 

5 

.005 

.943 

NS 

Base  X  Region 

5 

.003 

.946 

.05 

Estimated  Percent  Reduction  in 
Total  DOC  due  to  PSRO  Review 


Northeast    North  Central      South  West 
Percent  Reduction  -4.0%  -1.6%  +1.3%  -2.0% 
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limited  to  Northeast  trends,  however,  their  reduction  is  4.0%. 
Similarly,  when  PSROs  in  the  South  are  evaluated  against  national 
trends  they  evidence  no  effect  on  DOC;  however,  when  compared  with 
inactives  within  the  South,  they  actually  show  an  increase  in  DOC 
(+1.3%).  ~ 

Finally,  when  regional  variation  is  accounted  for,  the  overall 
reduction  in  DOC  due  to  PSRO  review  increases  from  1.58%  to  2.0%. 
There  are  legitimate  arguments  for  choosing  either  model  specifica- 
tion. It  was  felt,  for  the  purposes  of  this  report,  that  national 
estimates  had  more  universal  application  than  regional  estimates. 
Further,  the  national  evaluation  gives  a  more  conservative  estimate 
of  overall  PSRO  effect.  Thus,  it  seems  more  likely  that  the  nation- 
al evaluation  has  not  overestimated  the  impact  of  PSRO  review  on 
total  DOC. 

Hospital  Rate  Review  Analysis;  One  exogenous  variable  which 
was  not  included  in  the  original  regression  analyses  was  the  pres- 
ence or  absence  of  Hospital  Rate  Review  in  the  PSRO  area.  Because 
this  cost  containment  effort  might  also  affect  utilization  patterns, 
additional  analyses  were  run  to  test  for  this  effect.  There  are  two 
types  of  state  hospital  rate  review  agencies;  mandatory  and  volun- 
tary. /M  Regressions  were  run  using  just  the  mandatory  rate  setting 
areas  and  others  were  run  using  both  mandatory  and  voluntary  state 
agencies.  Only  the  mandatory  agencies  were  found  to  have  a  signif- 
icant impact  on  days  of  care.  In  the  seven  states  with  mandatory 
compliance  with  rate  setting,  there  were  40  active  PSROs  and  19 
inactive  PSROs.  Table  19  presents  the  results  of  this  regression 
run.  The  analyses  for  mandatory  rate  review  systems  can  be  sum- 
marized as  follows:  the  addition  of  a  Hospital  Rate  Review  variable 
to  the  analysis  does  not  change  the  impact  of  PSRO  review.  PSRO 
review  still  has  a  significant  effect  on  reducing  DOC.  There  is  no 
interaction  effect  between  PSRO  review  and  cost  containment  activ- 
ity. Rate  review  does,  however,  have  an  impact  on  DOC  independent 
of  PSRO  review.  The  general  effect  of  rate  review  is  to  increase 
DOC.  It  may  be  that  controlling  per  diem  charges  produces  an 
incentive  among  hospitals  to  increase  ALOS  and/or  DISC.  Increases 
in  either  of  these  would  enable  hospitals  to  maintain  revenues 
despite  controls  on  daily  charges. 


/_1  California,  Connecticut,  Maryland,  New  Jersey,  New  York,  Rhode 
Island  and  Washington  have  mandatory  rate  setting  compliance. 
Arizona,  Colorado,  Massachusetts,  Minnesota  and  Wisconsin  have 
voluntary  compliance  with  rate  setting.  All  were  operative  by 
January  1,  1977. 
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TABLE  19 


PROPORTIONAL  INCREMENTS  TO  EXPLAINED  VARIANCE 
AND  RAW  REGRESSION  COEFFICIENTS  FOR  REGRESSION  ON  TOTAL  DOC: 
SUPPLEMENTARY  ANALYSIS  INCLUDING  HOSPITAL  RATE  REVIEW 


Independent 
Variable 


Stage 


Increase 
in  R 


Cumulative 
R 


Significance 
Level 


Unstandardized 
Regression 
Coefficient 


Base  rate 
Adjustment 
Variables 

(a)  occupancy 

(b)  poverty 

(c)  density 

(d)  beds/1,000 

(e)  nursing 
beds/1, 000 

(f)  MDs/ 1,000 

(g)  %  aged 

(h)  %  DOC  due 
to  Medicare 

Hospital  Rate 

Review 
PSRO  Longevity 
PSRO  X  Rate 

Review 

Interaction 


.860 
.069 


.005 
.002 


.000 


,860 
,929 


,934 
,936 


.936, 


,001 
,001 


,001 
,05 


NS 


.8734 


2088.3385 
.4084 
.0044 
466.8973 

-16.8896 
-21.5672 
-8.1647 

-.0973 

145.5869 
-2.6041 


-2.2148 


Constant 


-1083.6370 


Program  Longevity  and  PSRO  Effectiveness;  The  analyses  of 
Section  2.2.4.2  indicate  that  there  is  no  incremental  effect  of  PSRO 
longevity  above  that  which  can  be  explained  by  the  dichotomous 
comparison  of  actives  (regardless  of  length  of  time  of  review)  and 
inactives.  Nevertheless,  it  was  felt  that  this  potential  relation- 
ship should  be  further  explored.  A  scattergram  of  percent  decrease 
in  DOC  versus  PSRO  longevity  is  presented  in  Figure  3.  This  plot 
suggests  an  age  effect  on  impact  since  low  values  of  longevity  (say, 
15  or  less  months  of  activity)  appear  connected  to  relatively  small, 
negative  percent  differences,  while  there  is  a  distinctly  greater 
variability  and  downward  trend  for  longevities  between  about  15  and 
30  months.  In  order  to  highlight  these  trends,  the  vertical  arrays 
of  points  in  the  scattergram  were  replaced  by  their  median  value? 
these  median  percent  difference  values  were  then  smoothed  using  the 
method  of  moving  medians.  The  resulting  smoothed  trend  line  (Figure 
4)    yields    some    interesting  observations.      PSROS  which  have  been 
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active  twelve  months  or  less  are  characterized  by  a  small,  negative 
percent  difference  (about  two-tenths  of  a  percent).  Those  active 
between  twelve  and  twenty-four  months  show  an  increasingly  favorable 
positive  difference  which  levels  off  at  about  3%  after  about  two 
years . 

Since  the  data  shown  in  Figures  3  and  4  are  cross-sectional 
(not  longitudinal),  it  is  impossible  to  infer  that  the  trend  shown 
in  longevity  would  also  show  up  in  time-series  data.  Although  the 
profile  is  consistent  with  the  hypothesis  of  a  time  effect  in  the 
impact  of  PSRO  review  procedures,  it  is  also  consistent  with  the 
hypothesis  that  the  self-selection  of  PSROs  with  respect  to  imple- 
mentation date  has  resulted  in  sites  with  the  greater  potential  for 
showing  an  effect  being  those  which  decided  to  implement  at  an 
earlier  date  (alternatively,  that  the  sites  with  the  least  potential 
delayed  implementation). 

The  possibility  of  differential  selection  within  the  "active" 
group  was  investigated  by  correlating  both  the  percentage  decrease 
in  DOC  and  the  program  variable,  longevity,  with  all  base  and 
adjustor  variables  for  these  96  sites.  The  only  variable  with  which 
both  decrease  in  DOC  and  longevity  correlated  significantly  was  the 
base  rate  (1974)  of  DOC.  The  median  smooth  of  the  scattergram 
relating  total  DOC  rates  for  1974  and  longevity  is  presented  in 
Figure  5.  It  is  apparent  that  the  cross-sectional  time  trend  in 
the  DOC  rates  is  similar  to  the  trend  found  in  the  percentage 
difference  measure  (Figure  4).  Thus,  a  reasonable  explanation  for 
the  trend  in  percentage  difference  values  is  that  the  relatively 
more  successful  PSROs  were  those  with  lower  DOC  rates  and  that  these 
sites  (for  other  reasons)  tended  to  begin  review  earlier  than  sites 
with  higher  DOC.  (The  correlation  between  Longevity  and  Total  DOC 
is  -.25,  while  that  between  percentage  decrease  in  DOC  and  Total  DOC 
for  1974  is  .21;  as  noted  above,  both  values  are  significant  at 
the  .05  level.)  Thus,  the  time-related  trend  in  percent  decrease  in 
DOC  values  appears  to  be  induced  by  the  fact  that  sites  with  lower 
DOC  rates  elected  to  implement  review  at  an  earlier  date  and  were 
relatively  more  successful  in  decreasing  DOC  rates. 

SNF-Certif ied  Days;  The  overall  weighted  average  saving  in 
days  of  care  due  to  PSRO  review  was  1.5%.  However,  it  can  be  argued 
that  the  unavailability  of  skilled  nursing  facility  (SNF)  beds 
restrains  PSROs  from  realizing  their  full  potential.  That  is,  in 
some  cases,  when  a  patient  is  certified  as  needing  care  at  the  SNF 
level  of  care,  the  patient  must  be  kept  in  an  acute  care  setting 
until  an  SNF  bed  becomes  available. 

To  get  some  indication  of  the  extent  of  this  problem,  the 
Office  of  Professional  Standards  Review  Organizations   (OPSR0)  asked 
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conditional  PSROs  to  report  information  related  to  certified  days . / 1 
Thirty-five  PSROs  responded.  Some  did  not  have  data  for  1977. 
Others  could  not  separate  SNF— certified  days  from  ICF  certified  days 
and/or  grace  and  administrative  days.  Nineteen  PSROs  did  have  data 
which  appeared  to  be  useable.    These  data  are  presented  in  Table  20. 

The  data  should  be  interpreted  with  extreme  caution.  The  data 
did  not  come  from  a  random  sample  of  PSROs;  it  reflects  only  those 
that  responded  with  apparently  good  data.  The  data  has  not  been 
validated  in  any  way  by  HSQB  central  staff  or  the  PSRO  evaluation 
staff.  The  extent  to  which  SNF  days  falling  into  the  "other" 
category, /2  were  under-  or  over-reported  is  unknown.  The  data  are 
presented  only  to  give  some  rough  estimate  of  the  magnitude  of 
SNF-certif ied  days. 

The  data  show  that  for  the  19  PSROs  which  responded  to  the 
request  and  for  which  1977  data  were  available,  the  average  percent 
of  total  acute  days  for  Medicare  patients,  which  were  SNF-certif ied 
was  1.9%.  The  average  percent  of  "other"  days  was  1.1%.  The 
range  of  SNF-certif  ied  days  was  from  .2%  to  9.5%.  Three  PSRO  had 
SNF-certif ied  days  of  4.0%  or  more.  "Other"  days  were  generally 
fewer  with  four  PSROs  having  2%  or  more. 

The  impact  estimates  for  these  PSROs  are  about  the  same  (1.5%) 
as  for  the  96  active  PSROs  as  a  group.  Generally,  there  appears  to 
be  little  relationship  between  SNF-certif ied  days  and  PSRO  impact. 
In  other  words,  the  existence  of  SNF-certif ied  patients  in  acute 
care  settings  apparently  did  not  affect  the  PSROs'  impact  on  lower- 
ing utilization. 

Nevertheless,  it  has  been  argued  that  SNF-certif  ied  days  paid 
at  the  acute  care  level  represent  a  potential  area  for  increased 
program  effect.  As  indicated,  the  regression  analysis  shows  that 
PSRO  program  activity  has  accounted  for  a  1.5%  reduction  in  days  of 
care;  SNF-certif  ied  days  paid  for  at  the  acute  care  level  account 
for  another  1.9%  of  total  days.  Given  these  figures  one  might 
expect  that,  given  an  adequate  supply  of  SNF  beds,  the  PSRO  effect 
might  have  been  as  great  as  3.4%  (1.5%  +  1.9%).  If  so,  it  would 
appear  that  there  is  unrealized  PSRO  potential  of  about  1.9%. 


/1  Memorandum,  September  30,  1978  to  conditional  PSROs  from  Direc- 
tor, OPSRO. 

/2  Other  days  are  those  days  used  which  were  neither  certified  at 
the  acute  nor  SNF  level  of  care.  They  include  grace  days, 
administrative  days,  and  other  non-certified  days. 
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TABLE  20 

SNF-CERTIFIED  AND  "OTHER"  DAYS  AS  A  PERCENT  OF  TOTAL 
REIMBURSED  ACUTE  CARE  HOSPITAL  DAYS  OF  MEDICARE  PATIENTS 


t  CI  V^Cll  8—     Ul      X  KJ  L.O.  X 

LJcxy  o    oil  j.    /  i 

rci  L» 

Ot-hor/2 

v  tllvi/  A 

Individual  PSRO 

T?         1  ma  4-  A 

RNDO  1 

1.2% 

0% 

.0% 

9CA04 

1.0% 

1.8% 

-1.2% 

1-9% 

2.1% 

A  *    1  D 

-1-4% 

•  WD 

3  o  V  D 

A  Ll  X  V  -J* 

9-4% 

.7  •  ■*  D 

.  1% 

•  ID 

-  1% 

•  ID 

iMAft  1 

■••Id 

All  1  \J  A 

6.9% 

VJ  •  7  D 

•  AD 

-4-1% 

**  •  ID 

<*/ 1  111  V  J 

.  9% 

•  AD 

-   -  3% 

i    5WI0 1 

2.2% 

.9% 

-1.8% 

or1  an  7 

1  9» 

•  AD 

— «\  Oft 

9CA00 

.2% 

.5% 

-2.3% 

4KY01 

.9% 

1.4% 

-  .3%  I 

10OR01 

1.1% 

.3% 

-5.1% 

8MT0  1 

1.7% 

1.4% 

-  .9% 

9CA09 

1.2% 

1.0% 

-  .3% 

3MD06 

2.7% 

.9% 

-  .0% 

2NY00 

4.0% 

.6% 

-5.3% 

2NJ02 

2.2% 

1.0% 

-  .4% 

1MA03 

2.4% 

3.7% 

-4.1% 

TOTAL/ 3 
(N=19) 

1.9% 

1.1% 

-1.5% 

/ 1  Percent  of  Total  Days  SNF-Certif ied;  These  are  days  of  hos- 
pital stay  deemed  medically  necessary  in  instances  where  the 
attending  physician  has  ordered  the  SNF  level  of  care,  the  PSRO 
had  certified  the  medical  necessity  of  the  SNF  level  of  care, 
and  where  the  payment  agency  provides  hospital  coverage  in 
instances  of  non-availability  of  SNF  beds. 

/2  Percent  Other;  These  are  the  total  days  of  acute  hospital  stay 
used  by  patients  beyond  the  period  certified  medically  neces- 
sary at  the  acute  or  skilled  nursing  level  of  care. 

/3    Totals  are  weighted  averages. 
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When  this  claim  is  made,  two  arguments  are  presented  in  re- 
buttal. First,  by  placing  a  patient  in  an  SNF  facility,  the 
total  cost  of  inpatient  days  is  not  saved.  There  is  the  substitu- 
tion of  SNF  care  for  acute  level  care.  The  total  savings  per  day  is 
not  the  cost  of  an  acute  day  but  at  most  the  difference  in  cost 
between  a  day  of  acute  care  and  a  day  of  SNF  care. 

Second,  Table  20  only  presents  the  data  from  the  active 
PSROs.  It  is  probable  that  in  inactive  PSROs  there  also  exist 
shortages  of  SNF  beds  and  that,  consequently,  there  are  patients  in 
these  areas  as  well  who  would  be  transferred  to  such  facilities  if 
SNF  beds  were  available.  Thus,  if  there  were  to  be  an  immediate 
rise  in  the  number  of  SNF  beds  in  both  active  and  inactive  areas, 
one  would  expect  patients  in  both  areas  to  be  transferred  to  SNF 
facilities.  This  reduction  would  be  due,  not  to  PSRO  review  activi- 
ties, but  to  the  exogenous  factor  of  availability  of  SNF  beds.  The 
relationship  of  PSRO  effect  to  SNF  availability  could  only  be  shown 
if  there  were  evidence  that  in  inactive  PSRO  areas  little  or  no 
effort  is  made  to  transfer  acute  care  patients  to  SNF  facilities 
when  it  is  medically  more  appropriate. 

All  things  considered,  it  is  not  clear  that  the  proportion  of 
total  days  certified  SNF  represent  pure  PSRO  effect  potential.  Only 
that  proportion  of  these  days  that  would  not  go  to  SNFs  if  there 
were  no  PSRO  can  be  considered  as  potential  PSRO  effect.  Accord- 
ingly, it  was  decided  not  to  include  SNF-certif ied  days  in  the 
estimate  of  PSRO  effect. 

2.2.4.4     Regression  Results  -  DISC,  ALPS,  Short  Stays  and 
Long  Stays 

As  previously  mentioned,  the  basic  analysis  was  performed  with 
respect  to  DOC.  However,  Discharges  per  1,000  Medicare  Enrollees 
(DISC),  ALOS  and  Short  and  Long  Stays  were  also  analyzed  to  help 
explain  the  DOC  findings*  This  section  briefly  discusses  regression 
analyses  related  to  these  variables. 

Table  21,  22,  and  23  present  the  summaries  of  explained 
variance  for  DISC,  ALOS  and  Short  and  Long  Stays,  respectively* 
Table  21  shows  that  the  results  of  the  DISC  analysis  parallel  the 
DOC  analysis.  PSROs  have  a  statistically  significant  impact  in 
reducing  DISC  in  the  first  half  of  the  year  but  no  effect  in  the 
second  half  of  the  year.  When  the  program  variable  is  expressed  as 
longevity,  PSRO  impact  is  significant  at  the  .01  level;  it  is 
significant  at  the  .05  level  when  expressed  as  intensity*  In  no 
instance  was  PSRO  review  associated  with  a  change  in  ALOS  (Table 
22).  Therefore,  the  change  in  DOC  is  apparently  due  to  the  PSRO 
effect  upon  discharges  (admissions)  rather  than  ALOS* 
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PSRO  review  could  be  expected  to  have  an  impact  on  either 
admissions  or  length  of  stay.  Admission  review  is  designed  to 
determine  the  appropriateness  of  admission  and  to  deny  payment  if 
hospitalization  is  unjustified.  Review  is  also  conducted  during 
hospital  stays  to  determine  the  appropriate  length  of  stay.  The 
comparative  findings  of  Tables  21  and  22  indicate  that  it  is 
admission  review  (which  has  an  impact  on  DISC)  rather  than  continued 
stay  review  (which  affects  ALOS)  which  is  more  effective.  This 
is  surprising  considering  the  program  management  data  given  in 
Section  6  in  which  admission  denials  are  seen  to  be  extremely  rare, 
constituting  only  about  one-tenth  of  one  percent  of  all  reviewed 
admissions.  One  explanation  may  be  that  the  existence  of  admission 
review  serves  as  a  deterrent  to  inappropriate  admissions:  physi- 
cians, aware  of  the  review  process,  may  be  screening  their  patients 
more  carefully  thereby  reducing  the  number  of  opportunities  for 
denial. 

The  percent  of  all  stays  that  are  short-stay  (0-3  days)  and  the 
percent  that  are  long-stay  (22+  days)  were  also  used  as  dependent 
variables  for  exploratory  purposes.  The  results,  presented  in 
Table  23,  show  that  PSRO  review  (when  expressed  as  longevity) 
is  associated  with  an  increase  in  the  percent  of  short-stays.  Thus, 
although  the  overall  ALOS  has  not  changed  due  to  PSRO  review,  there 
has   been   a   slight   shift   in   the   distribution  of    lengths   of  stay. 

2.2.4.5  Conclusions 

The  conclusions  of  this  study  are  as  follows: 

•  Rates  for  days  of  care  in  the  Medicare  population  were 
significantly  reduced  by  PSRO  review;  when  semi-annual 
comparisons  are  made,  this  result  applies  to  first  half-year 
data  only. 

•  The  Longevity  measure  (as  compared  with  the  Intensity 
measure)  is  a  better  indicator  of  the  impact  of  PSRO  review 
both  in  terms  of  the  amount  of  explained  variance  and  the 
estimated  percent-difference  measures. 

•  There  is  no  evidence  that  the  proportion  of  delegated 
review  is   associated  with   changes  in  the  outcome  measures. 

•  The  presence  or  absence  of  PSRO  review  is  the  primary 
predictor  of  PSRO  impact;  i.e.,  including  the  actual  number 
of  months  of  Longevity  or  percent  coverage  for  Intensity 
does  not  add  significantly  to  the  ability  to  predict  changes 
in  utilization. 

•  State-level  cost  containment  programs  raise  DOC  rates,  but 
have  no  demonstrable  effect  upon  the  impact  of  PSRO  review. 
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•  There  is  significant  regional  variation  with  respect  to  the 
benefits  of  PSRD  review. 

•  The  disaggregated  analysis  based  on  Longevity  yields  an 
estimate  of  -1.5%  as  the  average  change  in  DOC  rates  at- 
tributable to  PSFO  review. 

•  There  is  a  significant  decrease  in  discharge  rates  (about  3 
fewer  discharges  per  1  ,000  eligibles)  associated  with  PSRD 
review. 

•  Average  length  of  stay  is  not  influenced  by  PSRO  review, 
although  there  is  some  evidence  of  a  higher  proportion  of 
shorter  stays  when  review  is  present. 

2.3    Twenty  Percent  Medicare  Discharge  File  Study 

2.3.1  Objectives 

The  1977  PSRO  evaluation  compared  change  in  aggregate  hospital 
utilization  by  Medicare  enrollees  from  1974  through  1976  in  areas 
with  active  PSROs  to  similar  areas  with  inactive  PSROs  and  found 
that  the  difference  was  negligible.  This  study  considers  the 
possibility  that,  while  the  overall  effect  of  PSRO  may  have  been 
small,  the  program  reduced  the  average  length  of  stay  or  the  dis- 
charge rate  for  particular  diagnoses  or  procedures.  The  objective 
of  the  study  is  to  illustrate  an  approach  to  diagnostic  or  pro- 
cedure-specific evaluation  using  Medicare  data  through  1976.  It  is 
expected  that  the  approach  will  be  used  in  subsequent  evaluations  to 
examine  hypotheses  about  PSRO  effectiveness  in  reference  to  particu- 
lar diagnoses  or  procedures. 

For  illustration,  the  study  focused  upon  several  diagnoses  and 
procedures  that  have  been  discussed  in  the  health  care  literature  as 
being  subject  to  excess  hospital  care  with  respect  to  either  length 
of  stay  or  frequency  of  admission. 

2.3.2  Data 

The  following  data  sources  are  employed  in  this  evaluation: 

1 •  The  Twenty  Percent  Medicare  Discharge  File  contains  the  hospital 
discharge  records  of  20  percent  of  all  Medicare  enrollees.  The 
discharge  record  includes  the  identification  of  the  patient,  the 
patient's  county  of  residence,  and  the  diagnoses  and/or  procedures 
recorded  at  discharge.  From  this  file,  county-specific  discharge 
rates,  average  length  of  stay,  and  total  days  of  care  were  estimated 
for  selected  procedures  and  diagnoses  studied.  Data  for  the  years 
1971-1973  and  1976  were  used. 
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2.  The  Medicare  Master  Enrollment  File  covers  all  Medicare  eli- 
gibles  in  the  United  States.  The  use  of  this  file  and  the  Discharge 
File  enables  rates  to  be  calculated.  The  number  of  Medicare  aged 
eligibles  in  1974,  1975,  1976,  and  1977  were  22.0  million,  22.5 
million,  22.9  million,  and  23.5  million,  respectively. 

3.  The  Medicare  Provider  of  Service  File  contains  information  on 
inpatient  providers  qualified  for  Medicare  reimbursement.  By 
examining  the  date  when  each  provider  came  under  binding  PSRO 
review,  it  is  possible  to  estimate  the  level  of  review  (percent  of 
all  discharges  under  review)  for  any  PSRO  area  for  any  time  period 
in  the  study. 

4*  The  Master  Facility  Inventory  is  maintained  by  the  National 
Center  for  Health  Statistics  (NCHS)  and  contains  important  informa- 
tion on  supply  factors  (number  of  general  hospital  beds  and  nursing 
home  beds)  which  may  affect  Medicare  utilization  rates  independently 
of  any  PSRO  effect. 

5.  The  Area  Resource  File  contains  social  and  medical  environmental 
factors  which  also  need  to  be  controlled  for  in  the  analysis.  These 
include  population  density,  percent  of  families  with  yearly  incomes 
below  $5,000  and  total  number  of  physicians. 

2.3.3  Methods 

2.3.3.1  Summary  of  the  Approach 

For  the  procedures  and  diagnoses  selected,  geographic  areas 
were  identified  in  which  the  Medicare  population  had  a  comparatively 
high  ALOS  or  discharge  rate  in  1971-1973,  a  period  prior  to  the  PSRO 
Program.  Among  these  high  utilization  areas,  those  with  active  PSRO 
review  (more  than  half  of  the  discharges  under  review)  and  those 
with  inactive  PSRO  review  (less  than  five  percent  of  the  discharges 
under  review)  were  placed  into  two  groups.  To  evaluate  the  effect 
of  PSRO  on  utilization,  the  change  in  utilization  between  1971-1973 
and  1976  in  the  group  with  active  PSROs  was  compared  to  the  corres- 
ponding change  in  the  group  with  inactive  PSROs.  Adjustment  was 
made  for  differences  in  the  presence  of  determinants  of  hospital  use 
other  than  PSRO  between  the  two  groups. 

2.3.3.2  Selection  of  Diagnoses  and  Procedures  for  Evaluation 

The  assertion  that  care  for  individual  patient  problems  is 
excesssive  can  be  based  only  upon  comparison  of  care  rendered  with 
patient  indications.  Since  records  of  patient  indications  are  not 
available,  such  assertions  are  not  possible.  However,  the  care 
rendered  to  a  particular  population  may  be  suspected  of  being 
excessive,  if  similar  populations  are  known  to  receive  substantially 
less  care  for  similar  problems.  The  suspicion  is  strengthened  if 
there  is  evidence  that  the  additional  treatment  does  not  improve  the 
patient's  prognosis. 
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Regarding  ALOS,  acute  myocardial  infarction  was  selected 
because  ALOS  in  its  treatment  varies  substantially;  moreover, 
several  studies  have  shown  that  many  patients,  classified  as  "low 
risk,"  may  be  discharged  early  from  the  hospital  without  affecting 
their  prognosis »/ 1-6  Several  low  risk  surgical  procedures  were 
also  selected  for  evaluation  of  ALOS;  they  were  repair  of  inguinal 
hernia,  hemorrhoidectomy,  and  removal  of  cataract.  In  each  case, 
the  ALOS  varies  substantially  within  the  Medicare  population,  and 
the  procedures  are  of  sufficiently  low  risk  that  extending  the 
length   of   stay   is   not    likely   to  improve   the  patient's  prognosis. 

For  evaluation  of  the  effect  of  PSRO  on  discharge  rates, 
several  surgical  procedures  were  selected  including  cholecystectomy, 
removal  of  cataract,  prostatectomy,  repair  of  inguinal  hernia, 
hysterectomy,  and  hemorrhoidectomy.  Each  of  these  procedures  has 
been  discussed  in  one  or  more  studies  showing  variation  in  incidence 
rates  in  similar  populations  and  raising  the  possibility  that  high 
incidence  rates  represent  excessive  care./7-9 


/1_  Mather,  H.G.,  Peason,  N.G.  ,  Read,  K.L.Q.,  et  al:  Acute  myocar- 
dial infarction:  home  and  hospital  treatment.  Br.  Med.  J. 
3:334-338,  1971. 

/2  Groden,  B.M.,  ALlison,  A.,  Shaw,  G.B.:  Management  of  myocardial 
infarction:  the  effect  of  early  mobilization.  Scott  Med.  J. 
12:435-439,  1967. 

/3  Harper,  J.E.,  Kellett,  R.J.,  Connor,  W.T.,  et  al.  :  Controlled 
trial  of  early  mobilization  and  discharge  from  hospital  in 
uncomplicated  myocardial   infarction.      Lancet   2:1331-1334,  1971. 

/4  Hutter,  A.M.,  Jr.,  Sidel,  V.W. ,  Shine,  K.L.,  et  al:  Early 
hospital  discharge  after  myocardial  infarction.  N  Engl  J.  Med. 
288:1141-1144,  1973. 

/5  Hayes,  M.J.,  Morris,  G.K.,  Hampton,  J.R. :  Comparison  of  mobili- 
zation after  two  and  nine  days  in  uncomplicated  myocardial 
infarction.     Br.  Med.  J.  3:10-13,  1974. 

/6  McNeer,  J.F.,  Wagner,  G.S.,  Ginsburg,  P.B.,  et  al:  Hospital 
discharge  one  week  after  acute  myocardial  infarction.  N  Engl.  J. 
MEd.   298:     229-231,  1978. 

/7  Lewis,  C.E.:  Variations  in  the  incidence  of  surgery.  N.  Engl. 
J.  Med.   281:     880-884,  1969. 

/8  Bunker,  J. P.:  Surgical  manpower:  a  comparison  of  operations  and 
surgeons  in  the  United  States  and  in  England  and  Wales.  N.  Engl. 
J.  Med  282:     135-144,  1970. 

/9  Wennberg,  J.E.,  Gittlesohn,  A.M.:  Small  area  variations  in 
health  care  delivery.     Science  182:     110  2-1108,  1973. 
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2.3.3.3     Measurement  of  Utilization  and  Selection  of  Settings 

The  beneficiary-based  measures  of  hospital  utilization  were 
incidence  of  procedures  per  100,000  population  and  average  length  of 
stay.  Each  measurement  was  specific  to  the  population  of  Medicare 
enrollees  residing  within  county  boundaries.  County-specific, 
population-based  incidence  was  defined  as  the  number  of  Medicare 
enrollees  aged  65  and  over  residing  in  the  county  and  having  the 
care  of  interest  in  a  fixed  time  period,  divided  by  the  total  number 
of  such  enrollees  residing  in  the  county  in  the  same  period.  The 
hospital  care  did  not  have  to  be  provided  at  a  hospital  within  the 
county.  Similarly,  county-specific  population-based  ALOS  was 
defined  as  the  average  of  all  lengths  of  stay  of  patients  having  the 
hospital  care  of  interest  and  residing  in  the  county.  (Population 
based  total  days  of  care  is  the  product  of  population-based  inci- 
dence and  average  length  of  stay. ) 

The  evaluation  focuses  upon  the  effect  of  PSRO  on  reducing 
utilization  in  settings  that  had  high  use  prior  to  the  introduction 
of  the  program.  The  distribution  of  beneficiary-based  utilization 
was  estimated  for  Medicare  enrollees  during  1971  to  1973,  and 
counties  whose  population  of  Medicare  beneficiaries  fell  in  the 
upper  thirty  percent  of  the  utilization  distribution  were  included 
in  the  study. 

A  hospitalization  was  considered  to  be  under  PSRO  review  if  the 
admission  occurred  in  a  hospital  under  PSRO  review  and  the  date  of 
admission  was  subsequent  to  the  hospital's  date  of  binding  review. 
The  extent  of  PSRO  review  was  measured  by  the  proportion  of  dis- 
charges for  the  procedure  or  diagnosis  of  interest  under  review  for 
the  county's  population.  These  calculations  were  specific  to  each 
county . 

Aside  from  prior  utilization,  other  factors  were  considered  as 
potential  determinants  of  utilization.  These  included  demographic 
factors  such  as  age,  economic  factors  such  as  percent  of  popula- 
tion with  low  income,  and  supply  factors  such  as  hospital  beds  per 
100,000  population.  The  presence  of  such  factors  by  county  was 
defined  by  the  Area  Resource  File,  the  Master  Facility  Inventory, 
and  Medicare  enrollment  files.  A  list  of  factors  available  on  a 
county  basis  is  given  in  Table  24. 


100 


TABLE  24 


CHARACTERISTICS    OF    COUNTIES    THAT    MAY    AFFECT    HOSPITAL  UTILIZATION 


1 .  Mean  age  of  population  65  and  over 

2.  Sex  ratio  of  population  65  and  over 

3.  Population  per  square  mile 

4.  Percent  of  families  with  income  under  $5,000 

5.  Hospital  beds  per  100,000  population 

6.  Physicians  per  100,000  population 

7.  Nursing  home  beds  per  100,000  population 


2*3.3.4  Analysis 

The  effect  of  PSRO  review  on  ALOS  was  evaluated  by  testing  the 
null  hypothesis  of  no  association  between  PSRO  review  and  change  in 
ALOS  against  the  one-sided  alternative  of  an  association  between 
PSRO  review  and  a  reduction  in  ALOS.  To  test  this  hypothesis,  three 
analyses  were  conducted  with  reference  to  each  diagnosis  or  proce- 
dure selected.  First,  the  counties  in  the  upper  30  percent  of  the 
ALOS  distribution  for  1971-1973  were  grouped  into  active  (more  than 
half  under  review)  and  inactive  (less  than  five  percent  under 
review).  As  a  crude  approximation  of  the  effect  of  PSRO,  change  in 
ALOS  between  1971-1973  and  1976  was  compared  for  the  two  groups 
without  regard  for  the  presence  of  any  exogenous  factors  that  may 
have  affected  ALOS. 

The  second  analysis  accounted  for  the  fact  that  regression 
toward  the  mean  was  expected,  since  all  counties  had  high  use  in 
1971-1973.  Counties  with  active  PSROs  were  matched  with  inactive 
counties  having  the  same  length  of  stay  and  Medicare  enrollment  in 
1971-1973.  Change  in  ALOS  in  the  counties  with  PSROs  was  then 
compared  to  the  corresponding  change  in  the  matched  series  of 
counties  without  PSROs. 

To  control  for  other  factors,  regression  methods  were  used.  A 
linear  regression  ( 1 )  was  estimated  in  which  the  dependent  variable 
was  ALOS  for  1976,  and  the  independent  variables  were  the  ALOS  in 
1971-1973,  the  enrollment  in  1971-1973,  a  vector  of  possible  other 
determinants  of  ALOS,  and  the  proportion  of  discharges  under  PSRO 
review.    The  expression  takes  the  following  form: 
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(1)     y=  a  +  blXl  +  b2X2  t  b3X3  f  +  e 


where : 

y  =  ALOS  in  1976 

a  =  constant 

X1  =  ALOS  in  1971-1973 

X2  =  enrollment  in  1971-1973 

X^  =  vector  of  other  determinants  of  ALOS 

X^  =  the  proportion  of  discharges  under  PSRO  review  in  1976 
e  =  error 

b^,...,b4  =  regression  coefficients  corresponding  to.X^ 
through  X^  respectively. 

In  the  evaluation  of  the  effect  of  PSRO  on  discharge  rates, 
analogous  methods  were  used*  However,  in  the  regression  analysis, 
the  dependent  variable  was  the  logistic  transformation  of  the 
discharge   rate,    rather  than  the   discharge   rate   itself.      That  is: 

(2)  Logistic  transformation  of  the  discharge  rate  p  =  ln(p/l-p) 

The  logistic  transformation  is  desirable,  since  the  procedures 
and  diagnostic-specific  rates  being  analyzed  are  not  normally 
distributed  and  are  close  to  zero.  The  logistic  transformation  of 
the  discharge  rates  is  normally  distributed,  and  the  predicted 
values  are  bounded  between  zero  and  one.  If  the  rates  were  larger, 
as  they  tend  to  be  in  the  analysis  of  aggregate  discharge  rates, 
then  such  a  transformation  would  not  be  necessary. (The  transfor- 
mation is  not  required  for  ALOS,  since  ALOS  tends  to  be  normally 
distributed. ) 

A  consequence  of  the  logistic  approach  is  that  all  of  the 
county-specific  rates  used  in  the  regression  have  to  be  greater  than 
zero,  since  the  logarithm  of  zero  is  undefined.  To  accommodate  this 
restriction,  counties  with  less  than  2,000  Medicare  beneficiaries  in 
the  sample  were  excluded.  Counties  with  more  than  2,000  Medicare 
beneficiaries  but  zero  discharges  were  also  excluded.  All  exclu- 
sions were  made  without  knowledge  of  whether  the  county  had  active 
or  inactive  PSRO  in  1976. 


/1      Cox  DR:    Analysis  of  Binary  Data.      Methuen,    London,  1970 
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2*3.4  Limitations 


Several  limitations  of  the  study  deserve  consideration  in 
the  context  of  the  findings.  The  most  important  is  that,  at  least 
arguably,  there  may  not  have  been  any  excess  care  in  the  settings 
selected  for  study,  since  the  evidence  of  excess  is  only  a  priori 
suspicion  and  high  utilization.  If  there  were  no  excess  care  in 
these  areas,  then  failure  to  find  a  reduction  in  utilization  asso- 
ciated with  PSRO  review  need  not  reflect  an  absence  of  impact  in 
settings  of  actual  excess.  Another  limitation  is  that  exposure  to 
PSRO  in  1976  was  limited  to  a  small  fraction  of  the  population; 
therefore,  the  estimates  of  impact  may  not  be  generalizable  to  the 
entire  population.  A  third  limitation,  aside  from  sample  size,  is 
that  effective  work  by  the  PSRO  may  involve  a  learning  process,  so 
that  1976  may  have  been  too  early  to  estimate  its  effect.  Fourth, 
other  factors  in  addition  to  those  included  in  the  analysis  may 
affect  utilization.  Most  important  of  these  is  the  possibility  that 
other  review  activity   in  areas  without  PSRO  affected  utilization. 

2.3.5  Findings 

Length  of  Stay;  The  PSRO  Program  was  not  demonstrably  asso- 
ciated with  a  reduction  in  ALOS  for  the  diagnoses  and  procedures 
selected.  In  the  comparison  of  counties  with  active  and  inactive 
PSROs  matched  for  1971-1973  length  of  stay  and  enrollment,  the 
reduction  in  ALOS  between  1971-1973  and  1976  was  no  larger  in 
counties  with  PSROs;  in  fact,  there  was  slightly  less  reduction  for 
active  areas  (Table  25).  For  acute  myocardial  infarction  and 
repair  of  inguinal  hernia,  the  active  counties  reduced  ALOS  slightly 
more,  but  the  difference  was  not  significant.  PSRO  review  was 
actually  associated  with  relative  increases  in  ALOS  for  hemor- 
rhoidectomy and  cataract  removal.  In  the  regression  analysis,  PSRO 
review  was  not  associated  with  a  reduction  in  ALOS  for  any  of  the 
precedures  tested.  Moreover,  there  was  actually  a  significant 
association  between  PSRO  review  and  an  increase  in  ALOS  for  several 
of  those  procedures  (Table  26). 

Discharge  Rates;  PSRO  review  was  associated  with  a  reduction 
in  discharge  rates  between  1971-1973  and  1976  for  each  of  the 
providers  evaluated  in  the  regression  analysis;  however,  the  asso- 
ciation was  statistically  significant  (p  <  .05)  only  for  chole- 
cystectomy (Table  27).  In  the  matched  comparison,  there  was  a 
substantial  and  statistically  significant  relative  reduction  in 
the  discharge  rate  for  cataract  removal  (Table  28).  However, 
this  association  did  not  persist  in  the  regression  analysis. 

The  association  between  PSRO  review  and  the  reduction  in 
the  discharge  rate  for  cholecystectomy  will  be  a  hypothesis  for 
subsequent  evaluations.  Since  PSRO  review  was  negatively  associ- 
ated with  changes  in  discharge  rates  for  the  other  procedures 
selected,  they  will  also  be  studied  in  future  evaluations,  even 
though  the  association  was  not  significant  in  the  present  analy- 
sis. 
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TABLE  26 

REGRESSION  ANALYSIS  OF  EFFECT  OF  PSRO  ON  CHANGE  IN  ALOS  BETWEEN 
1971-1973  AND  1976  FOR  SELECTED  DIAGNOSES  AND  PROCEDURES. 


Diagnosis  or  Procedure 

Coefficient  of  PSRO 

Coefficient 

P 

1 .  Acute  myocardial  infarction 

+.024 

<.001 

2*  Repair  of  inguinal  hernia 

+.027 

<.001 

3.  Hemorrhoidectomy 

+  .043 

<.001 

4.  Removal  of  cataract 

+  .012 

<.001 

TABLE  27 

REGRESSION  ANALYSIS  OF  PSRO  REVIEW  ON  CHANGE  IN  THE  DISCHARGE 
RATE  FOR  SELECTED  PROCEDURES. 


Diagnosis  or  Procedure 

Coefficient  of 

PSRO 

Coefficient 

t  Value 

P 

1 •  Cholecystectomy 

-.0014 

2.28 

<.05 

2.  Removal  of  cataract 

-.0005 

1.29 

NS 

3*  Repair  of  inguinal  hernia 

-.0002 

0.37 

NS 

4.  Hysterectomy 

-.0001 

1.21 

NS 

5.  Hemorrhoidectomy 

-.0017 

1.61 

NS 
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2.4    MEDPAR/PHDDS  Study 
2.4.1  Objectives 

The  MEDPAR/PHDDS  analysis  was  designed  to  study  hospital 
utilization  by  Medicare  aged  beneficiaries  before  and  after  imple- 
mentation of  PSRO  concurrent  review.  Unlike  Use  Studies  1  and 
2,  which  use  data  generated  from  Medicare  administrative  records/ 
this  analysis  used  data  from  the  Medicare  20%  discharge  file  to 
represent  the  baseline,  i.e.,  pre-PSRO  period.  The  20%  discharge 
file  data  are  referred  to  hereinafter  as  the  MEDPAR  data,  after  the 
report  series  in  which  they  are  regularly  reported,  the  Medicare 
Provider  Analysis  and  Review  report.  Data  from  the  PHDDS  (PSRO 
Hospital  Discharge  Data  Set)  are  used  to  represent  the  post-PSRO 
period.  This  study  thus  represents  a  first  attempt  to  use  PHDDS 
data  for  assessment  of  program  impact. 

Seventeen  PSROs  are  included  in  the  analysis.  Selection 
was  based  upon  the  availability  of  at  least  one  calendar  year  of 
PHDDS  submissions  representing  a  period  of  100  percent  concurrent 
review.   (See  Table  29.) 

Two  major  issues  are  addressed  in  this  analysis.  The  first 
is  a  cross-validation  study.  Before  attempting  to  use  the  two  data 
sets  to  measure  program  impact,  it  was  essential  to  validate  their 
comparability.  During  1976,  the  only  overlapping  period  of  the  two 
data  sets,  seven  PSROs  conducting  100%  Medicare  review  had  submitted 
acceptable  PHDDS  data  for  the  entire  year  (see  Table  29).  Several 
measures  from  MEDPAR  and  PHDDS  were  compared  for  each  of  these  PSROs 
to  determine  the  extent  to  which  they  reflected  the  same  population 
and  utilization  experiences. 


TABLE  29 

PSROS  INCLUDED  IN  MEDPAR/PHDDS  STUDY 


PSROs 


100%   IMPLEMENTATION  DATE 


Eastern  Connecticut 

Passaic,  New  Jersey 

Rockland  County,  New  York 

National  Capitol,  District  of  Columbia 

So.  Central  Pennsylvania 

Southwestern  Pennsylvania 

Southeast  Wisconsin 

Utah* 

Idaho 

Multnomah ,  Ore  gon* 
Montgomery  County,  Maryland 
Western  Massachusetts 
Montana* 
Wyoming* 

Quad  River,  Illinois* 

GLSC,  Michigan* 

Charles  River,  Massachusetts* 


1Q 
2Q 
1Q 
3Q 
1Q 
1Q 
1Q 
2Q 
3Q 
4Q 
3Q 
1Q 
1Q 
3Q 
1Q 
1Q 
4Q 


1977 
1976 
1976 
1976 
1977 
1977 
1977 
1975 
1975 
1975 
1976 
1977 
1975 
1975 
1976 
1976 
1975 


*  PSROs  included  in  validation  study 
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Based  on  the  results  of  the  validation  study,  the  second 
study  is  a  Pre/Post  PSRO  Utilization  Study.  A  set  of  hypotheses 
were  developed  regarding  the  expected  impact  of  PSRO  concurrent 
review  activity  on  Medicare  hospital  use.  Each  hypothesis  was 
based  on  specific  functional  responsibilities  of  PSROs  and/or  known 
determinants  of  hospital  use  which  may  be  subject  to  PSRO  review  and 
inf luence . 

The  cross-validation  study  was  used  to  determine  which  of 
the  research  hypotheses  could  be  reliably  tested  with  the  MEDPAR/ 
PHDDS  data  sets.  Although  some  of  the  hypotheses  went  untested 
due  to  data  set  weakness,  all  the  hypotheses  are  presented  in 
this  report  since  they  represent  questions  which  should  ultimately 
be  addressed  in  a  comprehensive  PSRO  evaluation. 

2.4.2    MEDPAR/PHDDS  Cross-Validation  Study 

This  study  represents  the  first  use  of  PHDDS  as  an  evaluation 
tool.  It  was  considered  important  to  use  PHDDS  for  several  reasons* 
First,  PHDDS  data,  unlike  MEDPAR,  represent  all  Medicare  discharges 
rather  than  just  a  sample.  Second,  PHDDS  is  more  timely  than 
MEDPAR,  providing,  in  this  case,  1977  data  which  are  not  yet  avail- 
able from  the  Medicare  system.  Third,  PHDDS  data  are  generated  from 
a  medical  record  abstract  process  rather  than  the  billing  record  as 
are  Medicare  data,  and  are,  therefore,  not  as  susceptible  to  some 
of  the  reporting  problems  associated  with  the  billing  process. 
Fourth,  since  the  PHDDS  was  designed  specifically  to  serve  the  data 
needs  of  PSROs,  this  type  of  analysis  can  be  used  to  demonstrate  the 
current  and  potential  usefulness  of  PHDDS.  Fifth,  although  not 
considered  in  this  analysis,  PHDDS  contains  Medicaid  discharges, 
which  will  be  a  part  of  future  PSRO  evaluations. 

PHDDS,  however,  does  not  have  any  pre-program  or  non-program 
post-implementation  period  data.  It  was,  therefore,  necessary 
to  relate  PHDDS  to  a  comparable  data  set  which  contained  informa- 
tion on  the  pre-program  experience.  Given  the  desire  to  examine 
diagnosis-specific  data,  the  MEDPAR  file  was  chosen  to  represent  the 
pre-program  period  in  this  analysis. 

2.4.2.1    Fundamental  Differences 

The  two  data  sets  are  known  to  differ  in  several  ways.  MED- 
PAR contains  data  on  hospital  discharges  of  a  20  percent  sample 
of  Medicare  beneficiaries  aged  65  and  over  while  PHDDS  contains 
data  on  all  discharges  of  all  such  beneficiaries.  MEDPAR,  unlike 
PHDDS,  therefore  provides  estimates  of  hospital  use  which  are 
subject  to  sampling  variability  and,  especially  for  individual 
diagnoses  and/  or  procedures,  may  be  based  on  an  unreliably  small 
number  of  cases. 
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MEDPAR  contains  unique  hospital  identifiers  thus  permitting 
analysis  of  hospital-specific  use.  Although  the  PHDDS  are  unit 
record  data  and  can  be  aggregated  at  the  hospital  level,  hospital 
identifiers  are  confidential  so  they  cannot  be  linked  with  other 
data  sets.  There  is  also  the  possibility  that  some  hospitals  may 
not  be  represented  in  both  sets.  Attempts  were  made,  however,  to 
confirm  that  the  same  hospitals  were  included  in  both  the  MEDPAR  and 
PHDDS  data  used  in  this  study. 

Both  data  sets  suffer  from  missing  records  which  may  be  due 
to  different  reasons.  Lags  in  bill  processing  and  a  variety  of 
editing  checks  explain  much  of  the  MEDPAR  shortfall.  PHDDS  problems 
are  related  to  delays  in  reporting,  loss  of  abstracts,  and  with- 
holding of  records  which  do  not  clear  edit  checks.  The  extent  of 
these  shortfalls  and  their  influence  on  the  entire  data  set  (i.e., 
are    the    missing    records    atypical?)    are    not    clearly  understood. 

In  regard  to  the  use  of  diagnostic  specific  data,  the  defini- 
tion of  primary  diagnosis  is  known  to  differ  in  the  two  programs. 
The  Medicare  definition  is  "the  diagnosis  of  the  illness  or  condi- 
tion which  was  the  primary  reason  for  the  patient's  hospitaliza- 
tion". The  PHDDS  uses  the  Uniform  Hospital  Discharge  Data  Set 
definition  of  principal  diagnosis  as  "the  condition  established 
after  study  to  be  chiefly  responsible  for  occasioning  the  admission 
of  the  patient  to  the  hospital  for  care".  These  differing  defini- 
tions, confounded  by  the  involvement  of  different  types  of  personnel 
reporting  such  data  (billing  personnel  in  Medicare  versus  medical 
record  technicians  or  nurse  coordinators  in  PHDDS),  may  have  pro- 
duced discrepancies  in  the  selection  of  the  primary  diagnosis  from 
among  the  many  conditions  which  may  be  entered  in  hospital  records. 

Two  separate  validation  studies  of  the  Medicare  20%  discharge 
file  records  and  the  records  of  private  abstract  services  were 
conducted  by  the  Institute  of  Medicine  in  1977.  The  findings 
regarding  the  reliability  of  diagnostic  codes  were  similar  in  both 
studies:  a)  the  extent  of  discrepancy  between  medical  record  and 
file  data  varied  by  diagnosis  with  some  highly  reliable  and  others 
highly  discrepant;  b)  most  discrepancies  were  related  to  diagnosis 
selection  rather  than  coding  procedures;  and  c)  discrepancies  were 
reduced  when  higher  levels  of  case  aggregation,  e.g.,  3-digit 
rather  than  4-digit  codes  or  Diagnostic  Related  Groups  rather  than 
3-digit  diagnoses,  were  used. 

The  diagnostic  and  surgical  procedure  coding  often  differs 
between  the  two  sets;  MEDPAR  uses  ICDA-8  for  diagnoses  and  a  modi- 
fied CPT  for  procedures.  PHDDS,  however,  may  be  submitted  in 
either  H-ICDA-2  or  ICDA-8  for  both  items.  Conversion  of  H-ICDA-2  to 
ICDA-8  and/or  CPT  may  not  always  be  precise,  thus  hampering  the  use 
of   MEDPAR  and  PHDDS  together  for  condition-specific  analyses. 
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2.4.2.2    Analysis  of  Differences 


With  all  of  these  concerns  in  mind,  the  cross-validation 
study  was  designed  to  determine  the  comparability  of  MEDPAR  and 
PHDDS.  Data  for  seven  PSROs  which  were  100  percent  implemented 
and  reported  a  full  year  of  PHDDS  data  in  1976  were  used  in  this 
analysis.    The  following  variables  were  examined. 

Total  case  counts.  Differences  in  four  of  the  seven  areas 
were  not  significant.  Of  the  three  areas  with  significant  differ- 
ences in  total  case  count,  two  had  PHDDS  counts  somewhat  higher  than 
MEDPAR  while  the  third  had  a  MEDPAR  count  considerably  higher  than 
PHDDS.  In  the  aggregate,  the  seven  areas  had  131,  611  PHDDS  cases 
in  contrast  to  130,740  MEDPAR  cases. 

Average  length  of  stay  (ALPS),  1-40  day  stays  only.  There 
were  no  significant  differences  in  ALOS  for  stays  of  1  to  40  days 
between  the  MEDPAR  and  PHDDS  data  sets.  Stays  of  over  40  days 
were  eliminated  for  two  reasons.  One,  they  are  considered  atypi- 
cal of  short-stay  hospital  use.  Second,  it  is  likely  that  much 
of  the  shortfall  in  each  file  is  due  to  missing  records  of  over 
40  days  stays.  Those  long  stays  that  do  get  into  the  data  sets, 
therefore,  may  not  be  representative  of  that  class  of  stays.  This 
finding  suggests  that  pre/post  analysis  of  ALOS  using  the  two  data 
sets  may  be  justified. 

Age  distribution  of  cases  and  ALOS  (1-40  days).  In  five 
of  the  seven  PSROs,  there  were  no  significant  differences  in  the 
distribution  of  cases  by  age  groups  (65-69,  70-74,  75-79,  80-84, 
85+).  Of  the  two  areas  with  significant  differences,  one  differed 
primarily  among  those  over  80  years  of  age  while  the  second  had 
major  differences  among  those  under  75  and  over  85.  ALOS  for  1  to 
40  day  stays  was  similar  for  each  age  group  in  each  PSRO. 

Percentile  disribution  of  length  of  stay  (LOS).  The  10th, 

25th,  50th,  75th,  and  90th  percentile  stays  were  examined  using  all 
cases  in  each  PSRO.  Two  of  the  seven  had  virtually  identical 
distributions.  Four  of  the  remaining  five  differed  notably  only  in 
the  90th  percentile  stay.  In  almost  all  of  these  cases,  the  MEDPAR 
value  was  higher  than  PHDDS  perhaps  reflecting  different  "success 
rates"  in  incorporating  long  stays  (over  40  days)  into  their  files. 
Only  one  of  the  seven  PSROs  had  a  notable  difference  at  the  75th 
as  well  as  the  90th  percentile  stay. 

Admissions  of  1  to  40  day  stays  by  day  of  the  week  of  admis- 
sion. There  were  no  significant  differences  in  the  distribution  of  1 
to  40  day  stay  admissions  by  day  of  the  week  of  admission  in  five  of 
the  seven  PSROs. 

ALOS  (1  to  40  day  stays)  for  Friday  and  Saturday  admissions. 
ALOS  for  Friday  and  Saturday  admissions  were  similar  in  each  PSRO. 
Observed  differences  were  not  statistically  significant. 
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Discharges  of  live  patients  only,  1  to  40  day  stays  by  day 
of  the  week  of  discharge*  There  were  no  significant  differences 
in  the  distributions  of  these  discharges  in  five  of  the  seven 
PSROs .  Dead  discharges  were  eliminated  since  there  is  no  discre- 
tion in  determining  the  day  of  their  discharge. 

ALPS  (1  to  40  day  stays)  for  live  Monday  discharges.  There 
were  no  significant  differences  in  ALOS  for  these  cases. 

Distribution  of  surgical  and  non-surgical  cases.  There  were 
large  differences  between  MEDPAR  and  PHDDS  in  the  proportions 
of  cases  with  and  without  surgery.  This  discrepancy  apparently 
reveals  major  differences  in  the  coding  of  surgical  procedures, 
much  of  which  may  be  attributed  to  the  assignment  of  endoscopic 
procedures  to  surgical  rather  than  non-surgical  (diagnostic  test) 
classes.  Since  the  observed  case  distributions  were  discrepant,  ALOS 
within  these  categories  was  considered  unreliable. 

Diagnosis  and  surgical  procedure  data.  Case  counts  and  lengths 
of  stay  were  examined  for  15  diagnoses  and  14  surgical  procedures 
in  each  PSRO.  Conditions  were  selected  based  on  their  inclusion  in 
the  standard  MEDPAR  tabulations,  and  their  relative  coding  reli- 
ability based  on  the  IOM  validation  studies.  No  diagnosis  or 
procedure  had  similar  MEDPAR  and  PHDDS  values  on  both  proportion  of 
cases  and  ALOS.  Statistical  tests  applied  to  some  of  the  observed 
differences  showed  significance.  For  those  few  conditions  with 
similar  proportions  in  both  data  sets,  the  sample  size  was  too  small 
to  permit  reliable  testing  of  differences  in  ALOS. 

2.4.2.3    Comparison  Results 

It  was  clear  from  this  cross-validation  exercise  that,  in 
most  areas,  MEDPAR  and  PHDDS  files  contain  records  of  comparable 
numbers  and  types  (e.g.,  by  age  group)  of  Medicare  patients  con- 
suming a  comparable  number  of  days  of  care.  These  two  data  sets 
were,  therefore,  considered  comparable  for  testing  general  hypo- 
theses dealing  with  changes  in  overall  ALOS  and  case  distributional 
characteristics.  Data  on  clinical  features  of  the  hospital  stay, 
however,  were  not  considered  adequately  comparable  for  analysis. 
This  included  diagnoses,  surgical  procedures,  the  indication  of 
whether  or  not  surgery  was  performed,  and  the  presence  or  absence 
of  a  secondary  diagnosis.  Hypotheses  which  depend  on  these  data 
were,  therefore,  not  tested. 

The  1977  Medicare  100  percent  claims  file  was  also  used  to 
validate  the  comparability  of  the  two  data  sets  for  number  of 
discharges  and  average  length  of  stay  (the  only  two  variables 
available  from  that  file)  for  PSROs  included  in  the  1977  PHDDS. 
Since  MEDPAR  is  a  valid  random  sample  from  the  Medicare  100  per- 
cent file,    comparisons   of   the  Medicare    100   percent  file  with  the 
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PHDDS  file  for  Medicare  patients  are  generalizable  to  MEDPAR. 
Of  sixteen/ 1  PSROs  examined  (1977  data)  using  MEDPAR  and  PHDDS  ALOS, 
three  differed  by  0.5  days  or  more  and  3  differed  by  between  0.25 
and  0.5  days.  The  remaining  PSROs  exhibited  ALOS  within  0.25 
days. 

The  cross-validation  study  was  the  first  attempt  to  assess 
the  quality  of  PHDDS  data  in  relation  to  other  available  hospital 
discharge  data  sets.  (A  second  cross-validation  study  is  reported 
in  Section  2.5.)  The  PHDDS  performed  adequately  in  capturing 
general  information  about  the  patient  and  the  hospital  stay.  This 
analysis,  as  did  the  IOM  study,  raises  questions  about  the  quality 
of  some  data  items  in  the  PHDDS,  and  the  accuracy  and  completeness 
of  reporting  by  individual  PSROs. /2_  Future  improvements  in  compara- 
bility of  PSRO  and  Medicare  data  sets  may  result  from  a  combination 
of  several  efforts: 

1.  promulgation  of  uniform  definitions  among  data  systems 
(e.g.,  primary  diagnosis); 

2.  use  of  one  diagnostic  coding  system  (ICD-9-CM)  in  all 
systems ; 

3.  continued  development  of  quality  control  measures  within 
the  PHDDS  and  MEDPAR  systems  to  assure  accurate,  complete 
reporting  of  data. 

2.4.3    Pre/Post  PSRO  Analysis 
2.4.3.1  Hypotheses 

Based  upon  the  available  data  and  the  general  purpose  of 
the  PSRO  program  to  decrease  inappropriate  hospital  utilization, 
the  following  hypotheses  wore  initially  developed: 

H^:    ALOS  will   decrease  significantly  due  to  the  presence 
of  PSRO  review. 


/I  One  hundred  percent  Medicare  ALOS  was  not  available  for  one 
Massachusetts  PSRO. 

/2  This  is  similar  to  the  IOM  studies,  which  raised  questions 
about  the  accuracy  of  private  abstracting  services  and  the 
Medicare  abstracting  system. 
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H^:  PSRO  review  will  have  a  greater  impact  among  patients 
staying  1-40  days  than  among  those  staying  over  40  days. 

H^:  PSRO  review  will  influence  the  decrease  in  ALOS  among 
patients  aged  over  75  to  a  greater  degree  than  among 
those  under  75. 

♦H^s  PSRO  review  will  effect  a  greater  change  in  pre  and/or 
post-operative  ALOS  among  those  procedures  with  high 
pre  or  post-op  ALOS  in  1974  and  1975  as  compared  to 
MEDPAR  regional  norms. 

*H.:    PSRO  review  will  influence  the  decrease  in  ALOS  for 
4 

certain  diagnoses,  particularly  among  those  diagnoses 
with  high  ALOS  in  1974  and  1975  as  compared  to  MEDPAR 
regional  norms. 

H^A:  °SRO  review  will  be  associated  with  a  general  downward 
"shift"  in  the  ALOS  distribution  toward  lower  ALOS  and  a 
narrowing  of  the  distribution. 

H_   i     PSRO   review  will   be   associated  with  a   decrease  in  the 
5B 

proportxon  of  long  stays. 

H__:     PSRO   review  will  be   associated  with  a   decrease  in  the 
5C 

proportion  of  short  stays. 

*H^:  A  greater  proportion  of  cases  with  multiple  diagnoses 
will  be  present  after  review. 

*H^:  PSRO  review  will  influence  a  decrease  in  ALOS  to  a 
greater  degree  among  non-surgical  patients. 

*Hg:  Diagnostic  level  case  mix  will  not  change  due  to  the 
presence  of  PSRO  review.  (Changes  in  ALOS,  then,  cannot 
simply  be  attributed  to  changes  in  case  mix. ) 

H^:  PSRO  review  will  cause  a  decrease  in  the  proportion  of 
Friday    and   Saturday    admissions    and  Monday  discharges. 

H^q:  PSRO  review  will  have  the  greatest  effect,  on  decreasing 
ALOS  in  the  second  year  of  the  program,  on  those  over  75 
and  among  those  who  are  discharged  to  an  SNF.  (There 
will  be  a  correlation,  in  other  words,  between  change  in 
ALOS  and  the  availability  of  SNF  beds.) 


Due  to  the  findings  of  the  cross-validation  study,  hypotheses 
H3,  H4,  H6,  H7,  and  H8  were  not  tested. 
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2.4.3.2  Methods 


As  mentioned  above,  ALOS  and  various  distributional  type  vari- 
ables were  the  focus  of  the  longitudinal  study. 

Two  indicators  of  change  —  absolute  and  relative  —  were  used. 
The  relative  change  model  tests,  for  example,  the  null  hypothesis 
that  the  ratio  of  ALOS  in  1977  to  that  in  1976  is  equal  to  that 
between  1975  and  1974,  i.e. 


H  ALOS  77       =       ALOS  75 

:         ALOS  76  ALOS  74 


against  the  alternative  hypothesis  (H^)  that  the  ratio  has 
declined: 


H  ALOS  77  ALOS  75 

1         ALOS  76  ALOS  74 

The  null  hypothesis  states  the  expectation  given  no  PSRO 
effect;  the  alternative  hypothesis  states  the  expectation  given  a 
reduction  associated  with  PSRO  that  is  greater  than  in  the  pre-PSRO 
period. 

The  second  indicator  examines  absolute  rather  than  relative 
change.  In  the  case  of  ALOS,  the  null  hypothesis  is  expressed 
as : 

HQ :     ALOS  77  -  ALOS  76     =     ALOS  75  -  ALOS  74 


while  the  alternative  may  be  stated: 


H1 :     ALOS  77  -  ALOS  76   . <     ALOS  75  -  ALOS  74 

All  hypotheses  involving  ALOS  were  tested  using  these  two 
approaches.  This  included  changes  in  overall  ALOS,  ALOS  for  1-40 
day  stays,  ALOS  for  greater  than  40  days  stays  and  ALOS  for  dis- 
charges involving  patients  75  years  of  older. 

Hypotheses  concerning  shifts  in  overall  length-of-stay  dis- 
tribution were  evaluated  with  descriptive  statistics  including 
comparative  ogives  (cumulative  length-of-stay  distribution). 

Finally,  a  more  powerful  analysis  (three-way  analysis  of 
variance)  was  conducted  to  examine  early  PSRO  vs  late  PSRO  dif- 
ferences in  length  of  stay.  Factors  for  age  (>75  vs.  <75),  dis- 
position (discharged  home  vs.  discharged  to  SNF)  and  years  (1976  vs. 
1977)  were  examined  for  main  effects  as  well  as  interaction  effects 
on  length  of  stay. 


114 


2.4.3.3  Findings 


This  section  deals  with  those  longitudinal  ALOS  hypotheses 
(H1 ,H2 ,H5 ,H9 ,H1 0 )  whose  variables  were  basically  unaffected  by 
the  use  of  two  different  data  bases.  For  most  of  the  seven  PSROs 
involved  in  the  cross-validation  study,  non-significant  differences 
were  found  between  the  two  data  bases  for  these  variables:  over- 
all ALOS,  ALOS  for  long  stays,  ALOS  for  1-40  day  stays,  ALOS 
for  patients  greater  than  or  equal  to  75  years  old,  and  ALOS  for 
patients  less  than  75  years  old. 

This  analysis  is  limited  to  the  seventeen  PSROs  which  had  PHDDS 
data  for  1977.  Seven  of  these  were  included  in  the  validation  study 
using  1976  data  and  ten  had  1977  data  only   (see  Table  29,  p.  107). 

H  :  For  overall  ALOS,  most  PSROs  displayed  the  expected 
downward  trend.  However,  none  of  the  absolute  change  comparisons 
was  statistically  significant.  In  fact,  for  seven  of  the  ten  PSROs 
that  had  only  one  year  of  post  PSRO-data,  there  was  a  greater 
decline  in  ALOS  from  1974  to  1975  than  from  1976  to  1977.  For  PSROs 
that  were  100%  implemented  in  1976,  only  two  of  the  seven  had  a 
greater  decline  in  1975-1976  as  compared  to  1974-1975.  Although  two 
of  the  ten  PSROs  that  were  100%  implemented  in  1977  declined  over 
half  a  day  from  1976  to  1977,  this  effect  was  overshadowed  by  an 
inexplicable,  equally  large  or  larger  drop  in  the  1974-1975  period. 
One  PSRO  showed  no  effect  for  the  first  year  of  the  program,  but 
declined  over  one  day  in  the  second  year. 

H^:  For  ALOS  involving  over  40  day  stays,  five  of  nine  PSROs 
that  were  100%  implemented  in  1977  showed  a  sizeably  greater 
absolute  change  between  1976-1977  than  between  1974-1975.  For  those 
PSROs  100%  implemented  in  1976,  2  of  7  showed  a  sizable  decrease  in 
absolute  change  in  over  40  day  ALOS  for  the  first  year,  1975-1976, 
relative  to  the  base  period,  1974-1975.  Of  the  remaining  five  that 
did  not  decrease  in  the  first  year,  two  did  decrease  in  the  second 
year. 

For  the  1-40  day  stays,  six  PSROs  showed  a  greater  absolute 
change  between  1976-1977  than  between  1974-1975.  Nine  of  the 
remaining  PSROs  showed  some  absolute  decrease  in  ALOS  in  1977  but 
to  a  lesser  degree  than  the  pre-PSRO  period.  For  those  PSROs 
100%  implemented  in  1976,  three  showed  a  greater  decrease  between 
1975-1976  than  1974-1975.  Overall,  more  PSROs  experienced  a  de- 
crease in  ALOS  between  1975  and  1977  for  the  over  40  day  stays  than 
for  the  1-40  day  stays.  Eight  of  seventeen  PSROs  showed  a  greater 
change  in  ALOS  for  over  40  day  stays  than  for  1-40  day  stays  when 
comparing  1976-1977  against  1974-1975. 

H2:  Among  those  patients  aged  75  and  over,  fifteen  PSROs 
showed  some  decrease  in  ALOS  following  100  percent  implementation. 
When    examining    absolute    change    comparisons,    however,    only  seven 
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PSROs  showed  a  greater  decrease  in  ALOS  between  1976-1977  than 
between  1974-1975.  Five  of  the  seven  PSROs  with  1976  data  showed 
a  similar  impact  between  1975-1976  and  1974-1975. 

For  patients  aged  less  than  75,  thirteen  PSROs  displayed 
some  decrease  in  ALOS  between  1976-1977.  Of  these,  only  six  sites 
showed  a  greater  decrease  between  1976-1977  than  between  1974- 
1975.  For  those  PSROs  with  1976  PHDDS,  four  showed  a  greater 
decrease  in  ALOS  between  1975-1976  than  between  1974-1975. 

In  comparing  these  results,  PSROs  had  slightly  greater  impact 
on  reducing  ALOS  among  patients  75  or  older  than  among  those 
less   than   75.      This   was   not  statistically  significant,  however. 

H5A:  Wnen  examining  the  percentile  distribution  for  1974 
through  1977,  fourteen  of  seventeen  PSROs  showed  a 
decrease  of  at  least  one-half  day  at  the  25th  percentile 
after  the  implementation  of  review.  Nine  showed  a 
downward  shift  at  the  50th  percentile  and  thirteen 
decreased  at  the  75th  percentile. 

and  Hgp!  Tne  percent  of  short  and  long  stays  were 
examined  for  1974  through  1977  for  all  PSROs.  Stays  for 
live  patients  only  were  studied. 

Although  it  is  quite  clear  that  the  desired  impact  of  con- 
current review  is  to  decrease  the  percentage  of  long  stays,  the 
issue  of  short  stays  is  somewhat  ambiguous.  Very  short  stays 
(1  day)  are  most  likely  inappropriate  admissions.  Two  or  three 
day  stays  may  be  inappropriate  admissions  or  may  be  appropriate 
lengths  of  stay. 

Short  stays  are  generally  well  defined  by  diagnosis  or  pro- 
cedure such  as  T  and  A's  or  normal  deliveries.  Among  Medicare 
patients,  however,  there  are  fewer  diagnoses  or  procedures  that 
would  have  appropriate  short  stays. 

Despite  a  national  trend  of  increasing  short  stays,  one  PSRO 
showed  a  decrease  in  short  stays  after  review.  Six  of  the  re- 
maining sixteen  showed  a  decline  in  the  percentage  increase  of 
short  stays. 

The  percent  of  long  stays  was  expected  to  decrease  after 
the  implementation  of  concurrent  review.  Twelve  of  seventeen 
PSROs  displayed  decreasing  percentages  of  long  stays.  Five  PSROs 
showed  a  greater  decrease  between  1976-1977  than  between  1974-1975. 
Two  PSROs  100%  implemented  in  1976  showed  a  greater  decrease  between 
1975-1976  than  between  1974-1975.  Table  30  summarizes  the  declines 
and  increases,  by  year  and  hypothesis. 
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TABLE  30 

SUMMARY  OF  MEDPAR/PHDDS  TEST  RESULTS 


Measures  Tested 

1976  Data 

1977  Data 

N         Declines  Increases 

N         Declines  Increases 

H4.:  Total  ALOS 
1A 

H1  :  1-40  Day  ALOS 
>  40  Day  ALOS 

H  :     >  75  yr  old  ALOS 
<  75  yr  old  ALOS 

H5A:  25th  Percentile 
50th  Percentile 
75th  Percentile 

H^:  %  1-3  Day  Stays 
H5c:  %  41  +  Day  Stays 

7               3  4 

7               3  4 
7               2  5 

7               4  3 
7               5  2 

7               2  5 
7               3  4 
7               4  3 

7                1  6 

7                2  5 

17               3  14 

17               6  11 
17               7  10 

17               6  11 
17                7  10 

17              14  3 
17               9  8 
17               13  4 

17                1  16 
17               5  12 
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2.4.3.4  Conclusions 


In  general,  trends  in  length  of  stay  were  in  the  expected 
direction  although  not  statistically  significant.  However,  the 
downward  trend  in  length  of  stay  was  no  greater,  and  often  less, 
after  PSRO  implementation  than  it  had  been  before  PSRO  implementa- 
tion. 

An  important  contribution  of  this  analysis  was  the  PHDDS- 
MEDPAR  compatabi lity  study.  The  indications  from  this  analysis 
showed  that  at  higher  levels  of  aggregation,  the  two  data  sets 
were  comparable.  When  the  patients  were  disaggregated  to  the 
diagnosis/procedure  level,  however,  the  data  could  not  be  used 
for  pre-post  analysis.  Suggestions  were  made,  therefore,  for 
dealing  with  these  problems. 

In  addition  to  the  data  compatabi lity  study,  a  methodology 
for  a  pre-post  analysis  was  addressed.  Based  upon  the  validity 
study,  however,  many  of  the  hypotheses  put  forth  were  not  analyzed. 
It  is  felt  that  these  hypotheses  and  methodology  will  be  bene- 
ficial in  future  studies.  Statistically  significant  program  impact 
could  not  be  found.  However,  certain  trends  were  suggested. 
The  limitations  and  compatabi lity  issues  addressed,  however,  should 
be  used  in  conjunction  with  these  findings.  Considering  the  limited 
availability  of  data  and  relevant  data  elements,  the  findings 
of  impact  should  be  used  as  indicators  of  individual  PSRO  effect 
rather  than  a  statement  of  total  program  impact. 

2.5     PHDDS  Staging  Study 

2.5.1  Objectives 

As  part  of  the  1977  Office  of  Planning,  Evaluation  and  Legis- 
lation (OPEL)  evaluation  of  the  PSRO  program,  some  tentative  evi- 
dence surfaced  indicating  shifts  in  patient  mix  as  measured  by 
the  staging  index  of  disease  severity.  In  addition,  differential 
PSRO  impact  on  ALOS  was  observed  for  different  severity  levels 
of  certain  diagnoses.  For  some  diagnoses,  in  hospitals  conducting 
concurrent  review,  the  proportion  of  patients  in  Stage  1  decreased 
from  the  pre-PSRO  period  to  the  post-PSRO  period,  indicating  a 
possible  increase  in  the  severity  of  illness  for  patients  in  those 
particular  diagnostic  categories. 

It  is  possible  that  these  findings  reflect  more  appropriate 
use  of  the  hospital,  i.e.,  an  increased  proportion  of  Stage  1 
patients  were  treated  in  an  ambulatory  setting  thereby  reducing 
overall  ALOS.  It  is  also  possible,  however,  that  these  findings 
merely  reflect  changes  in  medical  record  keeping/abstracting  after 
the  implementation  of  PSRO  concurrent  review. 

To  explore  further  these  tentative  findings,  a  variety  of 
issues  are  investigated  in  this  present  study.  Using  the  PHDDS 
data  base   for    1976   and   1977,    the   following  issues   are  addressed: 
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Has  the  case  mix,  as  measured  by  the  staging  index,  of 
patients  treated  in  acute  care  hospitals  changed  from  1976 
to  1977  in  PSRO  hospitals? 

•  Have  diagnosis/stage-specific  admission  rates  changed 
from  1976  to  1977  in  PSRO  hospitals? 

•  Have  there  been  changes  in  stage-specific  ALOS  from  1976  to 
1977  and  have  these  changes,  if  any,  varied  as  a  function  of 
stage  of  illness? 

The  results  of  the  analyses  performed  to  address  these  issues 
are  presented  in  this  report.  Results  are  presented  separately 
for  Medicare  and  Medicaid  patients. 

In  addition  to  this  1976-1977  analysis  using  the  PHDDS  data 
base,  it  was  hoped  to  address  the  above  issues  on  a  pre/post- 
PSRO  basis  using  hospital  discharge  abstract  service  data  for 
the  "pre"  period  and  PHDDS  data  for  the  "post"  period.  Prior  to 
performing  this  pre/post  analysis,  however,  it  was  necessary  to 
establish  that  the  two  data  bases  are  comparable.  The  results  of 
that  validation  study  (Section  2.5.6)  indicate  a  lack  of  compar- 
ability between  the  two  data  bases,  precluding  the  performance  of 
the  pre/post  analysis  at  this  time. 

As  an  additional  aspect  of  the  present  project,  a  pilot  study 
is  being  undertaken  to  address  the  issue: 

•  Do  changes  in  medical  record  keeping  or  abstracting  pro- 
cedures since  the  advent  of  the  PSRO  program  account  for 
apparent  shifts   in  case  mix  as  measured  by  staging  index? 

This  reabs tract ing  study  is  still  in  progress. 

2.5.2  Data 

The  PSRO  Hospital  Discharge  Data  Set  (PHDDS)  served  as  the 
primary  data  source  for  this  project.  It  includes  all  PSROs  which 
were  100%  implemented  for  Medicare  review  for  the  entire  calendar 
years  of  either  1976  or  1977.  PHDDS  data  include  patient  dis- 
charge information  for  all  Federally-financed  patients.  The  PHDDS 
data  base  used  in  this  study  included  479,931  patient  discharges. 
These  records  represent  the  data  base  used  for  the  early/late 
analysis  on  eight  PSROs,  all  of  which  were  100%  implemented  for 
both  Medicare  and  Medicaid  review  in  1976  and  1977.  Newborns, 
Medicare  eligibles  under  age  65,  Title  V  discharges,  expired 
patients,  and  discharges  with  principal  payment  sources  other  than 
Medicare  or  Medicaid  were  dropped  from  the  analysis.  This  resulted 
in  96,904  staged  patient  records. 

It  was  further  decided  to  perform  separate  analyses  for  the 
Medicare    and   Medicaid    populations.       The    96,904    discharges  were 
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split  into  these  two  categories.  The  PHDDS  data  used  for  this  study 
consisted  of  70,704  staged  Medicare  patient  discharges  and  26,200 
staged  Medicaid  patient  discharges. 

In  addition,  variables  indicating  the  resource  characteristics 
of  the  PSRO  area  (such  as  short-term  beds  per  1,000  PSRO  popula- 
tion) were  derived  from  the  Bureau  of  Health  Manpower  Area  Resource 
File  (ARF) .  The  total  number  of  Medicare  enrollees  within  each  PSRO 
area  were  provided  by  HCFA  for  admission  rate  analyses. 

The  PHDDS  file  contains  patient  discharges  for  a  PSRO  only 
if  that  PSRO  was  fully  implemented  for  Medicare  review  for  the 
entire  calendar  year.  For  numerous  reasons,  data  from  various 
hospitals  within  specific  PSROs  arrived  late  for  processing.  Given 
the  time  constraints  of  the  study,  some  shortfalls  in  the  data 
remain.  For  example,  the  4th  quarter  of  1977  contained  few  dis- 
charges for  two  of  the  eight  PSROs.  One  hospital  from  a  PSRO  was 
dropped  from  the  analysis  entirely  because  of  faulty  data. 

2.5.3  Methods 

2.5.3.1    Disease  Staging 

It  is  clearly  desirable  to  deal  with  a  patient  population 
which  is  as  homogeneous  as  possible.  It  is  known,  for  example,  that 
an  elderly  patient  hospitalized  for  pneumonia  is  likely  to  need  a 
longer  stay  than  a  young  person  with  a  similar  diagnosis  because  the 
elderly  person's  overall  health  status  is  likely  to  be  more  fragile 
in  terms  of  recovery  time,  risk  of  complications,  and  other  factors. 
Such  normalizations  are  commonly  done  for  studies  such  as  this, 
and,  indeed  are  often  part  of  PSRO  LOS  criteria. 

In  recent  years,  however,  it  has  increasingly  come  to  be 
recognized  that  these  traditional  stratification  variables  are  not 
sufficient,  and  that  it  would  be  highly  desirable  to  include  a 
measure  of  illness  severity  in  the  normalization  process.  It  is 
clear  that  a  patient  who  is  "seriously"  ill  —  say,  a  burn  patient 
with  third-degree  involvement  over  60%  of  the  body  area  —  will 
experience  longer  stays  than  a  patient  who  is  less  seriously  ill  — 
say,  second-degree  burns  over  10%  of  body  area.  What  is  needed  is  a 
way  of  quantifying  this  concept  of  severity  so  that  it  can  be 
used  to  normalize  the  patient  mix. 

In  a  sense,  this  could  be  considered  an  extension  of  the 
precedent  of  stratifying  by  major  diagnosis.  The  severity  measure 
provides  a  more  sophisticated  way  of  taking  explicit  account  of 
clinical  characteristics  in  a  patient  population. 

The  staging  methodology  is  a  system  for  measuring  clinical 
severity.      It  evolved  from  the  clinical  staging  concept  developed 
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for  measuring  the  severity  of  various  forms  of  cancer.  The  staging 
system  is  now  more  developed  and  has  been  used  in  a  wide  range 
of  applications  from  quality  assurance  systems  to  outcome  measure- 
ments cost/severity  studies. 

Staging  is  essentially  a  method  of  refining  traditional  diag- 
nostic classifications  so  as  to  produce  predictive  clusters  useful 
for  evaluative  studies.  Staging  involves  defining  different  levels 
of  severity  for  specific  medical/surgical  problems.  The  method 
calls  for  separating  a  medical  problem  into  three  stages:  Stage  1 
—  a  condition  with  no  complications  or  with  problems  of  minimal 
severity;  Stage  2  —  a  condition  with  local  complications  or  prob- 
lems of  moderate  severity;  and  Stage  3  —  a  condition  with  systemic 
complications  or  problems  of  maximum  severity.  Sublevels  of  these 
stages  can  also  be  defined  to  allow  more  precise  definition  of  the 
staging  criteria.  As  an  example,  the  stage  definitions  for  appen- 
dicitis are  shown  in  Table  31. 

TABLE  31 
STAGE  DEFINITIONS:  APPENDICITIS 

Appendicitis 

A.  Appendicitis  with  perforation  leading  to 
localized  peritonitis 
or 

abscess  in  peritoneum 

B.  Appendicitis  with  perforation  leading  to 
a  subphrenic  abscess 

Problems  listed  in  Stage  2  plus: 

diffuse  peritonitis 
or 

septicemia  (sign  of  infection  in  one  or  more  organs) 
or 

shock  (hypotension,  oliguria,  obtundation,  signs  of 
peripheral  vascular  collapse) 

or 

intestinal  obstruction 
or 

pylephlebitis  with  or  without  liver  abscess 

Staging  classifications  have  been  developed  for  both  medical  and 
surgical  problems,  although  for  the  latter  the  definitions  are 
easier  to  develop  since  the  separation  of  a  primary  condition  (Stage 
1)  from  a  primary  condition  with  local  complications  (Stage  2)  is 
more  readily  apparent  than  in  a  medical  problem.  In  addition,  the 
system  can  also  work  for  a  medical  problem  with  more  than  one 
etiology,  such  as  diarrhea.  For  use  with  large  data  bases,  staging 
definition    has    been    translated   as    closely    as   possible    into  the 


STAGE  1: 
STAGE  2: 


STAGE  3: 
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equivalent  H-ICDA-2  and  ICDA-8  codes,  and  computer  programs  have 
been  developed  to  classify  patients  by  stage  based  on  information 
in  automated  hospital  discharge  abstracts  of  the  kind  used  for  this 
study. 

2.5 » 3. 2    Analytical  Techniques 

Multiple  linear  regression  techniques  using  binary  variables 
were  the  principal  methods  of  statistical  analyses.  This  type  of 
modeling  is  equivalent  to  an  analysis  of  variance  with  interactions 
in  traditional  statistics,  and  offers  a  simple  yet  versatile  method 
for  testing  many  complex  hypotheses. 


Basic  Regression  Model 


Since  a  principal  objective  was  to  determine  whether  or  not 
disease/stage-specific  ALOS  had  decreased  from  early  to  late  PSRO 
review  periods,  the  basic  regression  model  was  formulated  in  terms 
of  binary  variables  as  follows: 


LC^  =  bq  +  B1  Stage  +  B2  Time  +  B3  ( Stage  .Time )  +  e, 
where 


LOS^  =  The  length  of  stay  for  patient  i,  for  a  given  disease 
Stage  =  0  for  Stage  1  patients  of  a  given  disease. 
Stage  =  1  for  Stage  2/3  patients  of  a  given  disease. 


Time  =  0  for  1976,  i.e.,  early. 


Time  =  1  for  1977,  i.e.,  late. 

Stage »Time  =  A  cross-product,  or  interaction  term 


e  =  Stochastic  error  term. 


Bq ,  B^,  B^,  B^  =  Regression  coefficients  (to  be  estimated) 

As  usual,  it  is  assumed  that  the  expected  value  of  all  un- 
observed errors  is  zero  for  all  observations,  i.e.,  E(e)  =  0. 
If  so,  it  can  be  shown  that: 


ALOS  Stage  1,   1976    =  BQ 

ALOS  Stage  2/3,   1976    =    BQ  + 

ALOS  Stage  1,   1977     =     BQ  +  B2 

ALOS  Stage  2/3,    1977     =     BQ  +  B1  +  B2  +  B3 

where  ALOS  denotes  the  mean  LOS  for  all  patients  having  similar 
characteristics . 
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A  diagrammatic  representation  of  these  coefficients  is  as 

follows:  .  

1976  1977  (1977-1976) 


ALOS,  Stage  1 


B 


0 


B 


2 


ALOS,  Stage  2/3 


1 


VVVB3 


A(  Stage  2/3-Stage  1) 


B 


1 


B..+B 


3 


B 


3 


The  key  hypotheses  to  be  tested  here  are  the  significance  levels  of 
B_  and  B^  +  B_.  They  measure  the  longitudinal  differences  in 
stage-specific  "XOS  between  the  early  versus  late  review  period  for 
each  disease.  In  addition,  B^  alone  will  indicate  any  significant 
differences  between  the  changes  in  ALOS  for  Stage  2/3  patients 
contrasted  with  changes  for  Stage  1  patients. 

Interaction  of  Stage  Distribution  and  PSRO  Area  Supply  Char- 
acteristic Variables 

The  basic  regression  equation  can  be  modified  to  include 
the  influence  of  selected  PSRO  resource  or  supply  characteristic 
variables  on  the  relationship  between  ALOS,  severity  of  illness  and 
time.  It  is  desirable  to  test  the  impact,  upon  ALOS,  of  such  vari- 
ables as  number  of  short-term  beds  per  1,000  PSRO  population,  number 
of  MDs  per  1,000  PSRO  population  and  the  availability  of  extended 
care  beds  per  1  ,000  population  aged  65  and  over.  To  do  so,  the 
basic  regression  model  is  generalized  as  follows: 

Y.  =  B    +  B  X  +  B    Stage  +  B  Time  +  B  ( X  •  Stage) 


where  in  addition  to  terms  previously  defined: 

Y^  =  ALOS  for  a  given  disease. 

X  =  A  PSRO  area  supply  characteristic  variable;  for 
example  extended  care  beds  per  1,000  population 
aged  65  and  over. 

Z   ,   z     =  Other  independent  variables,  characteristic 
of  the  PSRO. 

X  .  Stage  =  A  second-order  interaction  term. 

X*  Stage  •  Time  =  A  third-order  interaction  term. 


o 


+  B     (X  •  Time)   +  B .( Stage  •  Time)   +  B_(X  •  Stage  •  Time) 

O  D  7 
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Thus,  one  PSRO  supply  characteristic  interacting  with  stage 
distribution  over  time  my  be  modeled*  In  addition  to  measuring 
these  differential  effects  of  a  single  PSRO  supply  characteristic  on 
relative  changes  in  ALOS,  the  Z  type  independent  variables  will 
simultaneously  control  for  the  impact  of  other  PSRO  characteristics* 
While  both  X  and  the  Z  •  s  are  usually  continous  variables,  to  aid 
in  the  interpretation  or  the  coefficients  they  will  be  dichotomized: 
each  will  have  a  value  of  zero  when  the  characteristic  is  small  in 
value  or  is  absent,  and  a  value  of  one  when  the  characteristic  has  a 
large  value  or  is  present*  Following  the  procedure  developed  above , 
it  can  be  shown  that  the  meaning  of  the  coefficients  is  as  follows: 

Bq  =    ALOS  Lo  X,  Stage  1,  1976 

«    ALOS  (Hi  X  -  Lo  X,  Stage  1,  1976) 

B2  =    ALOS  (Stage  2/3  -  Stage  1,  Lo  X,  1976) 

B3  =    ALOS  (1977-1976,  Lo  X,  Stage  1) 

B    =     ALOS  (Hi  X-Lo  X,  Stage  2/3,   1976)  -    ALOS  (Hi  X-Lo  X, 
Stage  1,  1976) 

B5  =     ALOS  (  1977-1976,  Hi  X,  Stage   1)   -  ALOS  (1977-1976,  Lo  X, 
Stage  1 ) 

Bg  =    ALOS  (1977-1976,  Lo  X,  Stage  2/3)  -  ALOS  (1977-1976,  Lo  X, 

Stage  1 ) 

B?  =    ALOS  [1977-1976  (Hi  X,  Stage  2/3)     -  (Lo  X,  Stage  2/3)]  - 
ALOS [1977-1976     (Hi  X,  Stage  1)  -  (Lo  X,  Stage  1)1 

where  ALOS  indicates  the  mean  LOS  for  all  patients  having  similar 
characteristics,  Hi  X  denotes  a  PSRO  with  a  large  supply,  and  Lo  X 
a  PSRO  with  a  small  supply  of  the  resource  in  question* 

This  specification  enables  many  complex  hypotheses  to  be 
tested.  In  light  of  the  added  hypothesis  to  be  tested  here,  i.e., 
that  PSRO  area  resources  are  associated  with  changes  in  LOS  for 
given  stages  of  a  disease,  three  coefficients  are  of  major  interest 
to  this  study: 

•  BR  measures  for  Stage  1  patients  whether  or  not  the  change 
in  length  of  stay  (between  the  early  and  late  review  period) 
was  different  in  high  as  compared  to  low  supply  areas*  For 
example,  if  Stage  1  patients  had  a  greater  reduction  ( 1977- 
1966)  in  LOS  in  high  supply  PSROs  then  in  low  supply  PSROs, 
B    would  display  a  negative  sign. 
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•  The  sum  of  +  tests  the  same  hypothesis  as  above 
for  Stage  2/3  patients. 

•  B  measures  whether  the  difference  in  the  change  in  ALOS 
was  greater  for  Stage  2/3  patients  treated  in  high  versus 
low  supply  areas.  It  is  also  correct  to  say  that  B^  mea- 
sures whether  or  not  the  difference  in  the  change  in  ALOS 
between  Stage  2/3  and  Stage  1  patients  was  dependent  on 
PSRO  area  resource  supply. 

This  more  complex  regression  equation  is  capable  of  estimating 
the  magnitude  of  the  difference  in  ALOS  between  stages  of  illness 
while  controlling  for  PSRO  area  supply  characteristics.  Furthermore, 
this  approach  provides  additional  information  for  assessing  the  PSRO 
review  program's  impact  on  hospital  utilization. 

2.5.4    Medicare  Utilization  Analysis 

In  this  section  the  analysis  and  findings  related  to  changes 
in  hospital  utlization  patterns  for  Medicare  recipients  are  pre- 
sented. Patients  were  assigned  to  stages  based  on  the  diagnostic 
codes  used  in  the  PHDDS  data  system.  An  arbitrary  minimum  of 
100  patients  per  diagnosis  was  established.  After  dropping  from 
consideration  the  diagnoses  which  did  not  meet  this  criterion,  the 
27  remaining  diagnoses,  representing  69,826  patients,  were  used  for 
the  analysis  described  in  this  chapter.  The  27  diagnoses  used  in 
the  analyses  are  listed  in  Table  32. 

2.5.4.1    Analysis  of  Changes  in  Diagnostic  Case  Mix 

The  following  null  hypothesis  was  tested  using  a  Chi-Square 
test  for  each  of  the  27  diagnoses  included  in  the  analysis: 

Hq  :    There   has   been   no   change   in   the  proportion   of  patients 
by  stage  from  1976  to  1977. 

For  purposes  of  running  the  Chi-Square  test,  Stage  2  and  3 
patients  of  each  diagnosis  were  combined.  An  example  of  the  test 
as  run  for  hypertension  is  shown  in  Table  33.  As  can  be  seen  from 
this  table,  in  1976,  45%  of  the  hypertension  admissions  were  Stage 
1,  while  in  1977,  38%  of  the  hypertension  admissions  were  Stage  1. 
This  decrease  in  percent  of  Stage  1  admissions  is  statistically 
significant  and  implies  a  move  toward  a  more  complex  case  mix  for 
hypertension   patients    in    the    PSRO    areas    included   in   the  study. 
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TABLE  32 

DIAGNOSES  USED  IN  STAGING  ANALYSIS 

Diagnoses 

Alcoholism 

Appendi  ci t i  s 

Bacterial  Pneumonia 

Benign  Prostatic  Hypertrophy 

Bronchial  Asthma 

Bronchopneumonia 

Carcinoma ,  Breast 

Carcinoma/  Cervix 

Carcinoma,  Prostate 

Cholecystitis 

Cirrhosis  of  Liver 

Colon  Carinoma 

Diabetes  Mellitus 


OF  MEDICARE  PATIENTS 

Diarrhea  Gastritis 
Diverticulitis 
External  Hernia 
Gout 

Hypertension 
Myocardial  Infarction 
Pancreatitis 
Peptic  Ulcer 
Pneumonia,  Unspecified 
Renal  Calculus 
Rheumatoid  Arthritis 
Thrombophlebitis 
Urinary  Tract  Infection 
Viral  Pneumonia 
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TABLE  33 
STAGED  DIAGNOSIS:  HYPERTENSION 


1976  1977 


Stage  1 

1 ,099 

954 

(45%) 

(38%) 

Stage  2/3 

1 ,335 

1,566 

(55%) 

(62%) 

2,434  2,520 


Chi  Square  =  27.150 
Degrees  of  Freedom  =  1 
p  <  .0001 


A  similar  pattern  of  a  shift  toward  a  more  complex  case  mix 
from  1976  to  1977  was  observed  for  six  diagnoses  in  addition  to 
hypertension.  The  shifts  for  colon  carcinoma  and  diverticulitis 
were  statistically  significant  at  the  .05  level  while  the  shifts 
for  diabetes  mellitus,  peptic  ulcer  disease,  rheumatoid  arthritis 
and  thrombophlebitis  were  significant  at  the  .10  level. 

For  only  one  diagnosis,  carcinoma  of  the  breast,  was  a  stat- 
istically significant  shift  found  in  the  direction  of  a  less  com- 
plex case  mix.  This  is  thought  to  be  an  important  change,  in 
that  early  diagnosis  and  treatment  is  especially  important  in  the 
treatment  of  this  disease.  The  other  nineteen  diagnoses  did  not 
change  in  case  mix  from  1976  to  1977. 

While  the  null  hypothesis  of  no  shifts  in  stage  distribution 
can  be  rejected  for  eight  of  the  diagnoses,  a  number  of  explanations 
are  possible  for  this  phenomenon.  PSRO  review  activity,  especially 
the  admission  certification  process,  may  be  having  the  desired 
effect  of  reducing  the  number  of  unnecessary  Stage  1  admissions. 
An  additional  possible  explanation  however  is  that  medical  record 
keeping  and/or  abstracting  practices  have  changed  toward  the  record- 
ing of  additional  patient  complications.  Such  a  change,  which  may 
in  itself  be  prompted  by  PSRO  review,  could  make  it  appear  that  more 
appropriate  use  of  the  hospital  is  occurring  while  in  fact  the 
apparent  shifts  are  merely  data  artifacts.  This  alternate  explana- 
tion cannot  be  assessed  until  the  re-abstracting  study  is  completed. 
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2*4.5*2    Admission  Rate  Analysis 


A  diagnosis/stage-specific  admission  rate  analysis  was  per- 
formed using  total  Medicare  eligibility  data  for  the  PSRO  areas 
included  in  the  sample.     The  following  null  hypothesis  was  tested: 

Hq  :      There  have  been  no  changes  in  the  stage-specific  admis- 
sion rates  from  1976  to  1977. 

Significant  decreases  in  the  admission  rates  for  Stage  1 
patients  were  observed  in  five  diagnoses:  bronchopneumonia,  diver- 
ticulitis, hypertension,  viral  pneumonia  and  appendicitis  while 
significant  increases  were  observed  in  two  diagnoses:  benign 
prostatic  hypertrophy  and  renal  calculus. 

For  Stage  2  and  3  (2/3)  patients,  a  significant  decrease  in 
admission  rate  was  observed  for  carcinoma  of  the  breast  patients 
while  significant  increases  in  admission  rates  were  observed  for 
colon  carcinoma,  diabetes  mellitus,  hypertension,  peptic  ulcer 
disease  and  thrombophlebitis  patients. 

The  results  of  the  analysis  of  changes  in  diagnostic  mix  are 
summarized  in  Table  34.  Potential  reasons  for  those  changes  may 
be  observed  from  the  results  of  the  admission  rate  analysis  sum- 
marized in  the  bottom  part  of  the  table. 

It  will  be  seen  that  for  all  five  of  the  conditions  showing  a 
significant  increase  in  Stage  2/3  admissions  per  1,000  eligibles 
(colon  carcinoma,  diabetes  mellitus,  hypertension,  peptic  ulcer 
disease  and  thrombophlebitis),  there  was  also  a  significantly  more 
complex  case  mix.  In  addition,  a  decrease  in  Stage  2/3  admissions/ 
1,000  eligibles  for  carcinoma  of  the  breast  was  accompanied  by 
a  less  complex  case  mix.  These  changes  may  or  may  not  be  due  to  the 
impact  of  PSRO  review.  The  increase  in  the  State  2/3  admission  rate 
and  the  resulting  shift  to  a  more  complex  case  mix  for  colon  car- 
cinoma, for  example,  may  be  attributed  to  an  increased  longevity  for 
these  patients. 

Diverticulitis  is  a  condition  for  which  it  can  be  argued 
that  the  effect  of  PSRO  review  was  to  decrease  the  Stage  1  admission 
rate,  without  increasing  the  Stage  2/3  admission  rate,  the  overall 
result  being  a  shift  to  a  more  complex  case  mix.  (Hospitalization 
is  not  always  required  for  Stage  1  diverticulitis.)  Peptic  ulcer 
disease,  however,  is  a  good  counter  example.  The  shift  to  a  more 
complex  case  mix  for  peptic  ulcer  disease  was  apparently  due  solely 
to  the  significant  increase  in  Stage  2/3  admission  rate. 

Finally,  hypertension  showed  both  a  significant  increase 
in  the  Stage  2/3  admission  rate  and  a  significant  decrease  in  the 
Stage  1  admission  rate,  resulting  in  a  shift  to  a  more  complex  case 
mix. 
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Table  34 


MEDICARE  STAGING  ANALYSIS  BY  DIAGNOSIS: 


SUMMARY  OF  FINDINGS 


A. 


Changes  in  Stage  Distribution  from  1976  to  1977 


Less  Complex  ( 1 ) 


Carcinoma  of  the  Breast** 


More  Complex  (7) 


Colon  Carcinoma** 


Diabetes  Mellitus* 
Diverticulitis  ** 
Hypertension** 
Peptic  Ulcer  Disease* 
Rheumatoid  Arthritis* 
Thrombophlebitis* 


Appendicitis 

Bacterial  Pneumonia 

Benign  Prostatic  Hypertrophy 

Bronchial  Asthma 

Bronchopneumonia 

Carcinoma  of  the  Cervix 

Carcinoma  of  the  Prostate 

Cholecystitis 

Cirrhosis  of  the  Liver 

Diarrhea  Gastritis 

External  Hernia 

Gout 

Myocardial  Infarction 
Pancreatitis 
Pneumonia,  Unspecified 
Renal  Calculus 
Urinary  Tract  Infection 
Viral  Pneumonia 


No  Change  (19) 


Alcoholism 


*  Significant  at  .10  level 
♦♦Significant  at  .05  level 
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Table  34  -  Continued 
MEDICARE  STAGING  ANALYSIS  BY  DIAGNOSIS:     SUMMARY  OF  FINDINGS 


B.      Changes  in  Admission  Rates,  1976  to  1977 


Stage  1 ; 


Increases  (4) 


Decreases  (5) 


Benign  Prostatic  Hypertrophy** 
External  Hernia* 
Myocardial  Infarction* 
Renal  Calculus** 

Appendicitis* 
Bronchopneumonia** 
Diverticulitis** 
Hypertension** 
Viral  Pneumonia** 


No  Change  (18) 


Stage  2/3; 


Increases  (5) 


Decreases  ( 1 ) 
No  Change  (21) 


*  Significant  at  .10  level 
♦•Slgnif leant  at  .05  level 


Alcoholism 
Bacterial  Pneumonia 
Bronchial  Asthma 
Carcinoma  of  the  Breast 
Carcinoma  of  the  Cervix 
Carcinoma  of  the  Prostate 
Cholecystitis 
Cirrhosis  of  the  Liver 
Colon  Carcinoma 
Diabetes  Mellitus 
Diarrhea  Gastritis 
Gout 

Pancreatitis 

Peptic  Ulcer  Disease 

Pneumonia,  Unspecified 

Rheumatoid  Arthritis 

Thromboplebitis 

Urinary  Tract  Infection 

Colon  Carcinoma** 

Diabetes  Mellitus* 

Hypertension** 

Peptic  Ulcer  Disease** 

Thrombophlebitis* 

Carcinoma  of  the  Breast* 

Alcoholism 

Appendicitis 

Bacterial  Pneumonia 

Benign  Prostatic  Hypertrophy 
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Table  34  -  Continued 


MEDICARE  STAGING  ANALYSIS  BY  DIAGNOSIS:     SUMMARY  OF  FINDINGS 


Changes  in  Admission  Rates,   1976  to  1977  -  continued 

Bronchial  Asthma 
Bronchial  Pneumonia 
Carcinoma  of  the  Cervix 
Carcinoma  of  the  Prostate 
Cholecystitis 
Cirrhosis  of  the  Liver 
Diarrhea  Gastritis 
Diverticulitis 
External  Hernia 
Gout 

Myocardial  Infarction 
Pancreatitis 
Pneumonia,  Unspecified 
Renal  Calculus 
Rheumatoid  Arthritis 
Urinary  Tract  Infection 
Viral  Pneumonia 
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2.5.4,3    Length  of  Stay  Analysis 


Using  the  regression  model  described  earlier,  the  statistical 
significance  of  observed  changes  in  diagnosis/stage-specific  mean 
length  of   stay  was   determined.      Two  null  hypotheses  were  tested: 


H_:  There  has  been  no  change  in  stage-specific  ALOS  from  1976 
to  1977. 


Hq  :  There  have   been  no   differences   in  changes   in  ALOS  as  a 
function  of  stage  of  illness. 


The  results  of  the  analysis  are  displayed  in  Table  35. 

To  reiterate  the  interpretation  of  the  key  regression  coef- 
ficients, represents  the  difference,  1977-1976,  in  ALOS  for 
Stage  1  patients.  The  sum  +  represents  a  similar  statistic 
for  Stage  2/3  patients.  The  B  coefficient  represents  the  dif- 
ference between  the  change  in  ALOS  for  Stage  2/3  patients  and 
the  change  in  ALOS  for  Stage  1  patients.  Therefore,  a  statis- 
tically significant  negative  value  of  B_  implies  a  significantly 
greater  decrease  in  mean  LOS  for  Stage  2/3  patients  than  for  Stage  1 
patients. 

Reviewing  the  results  in  Table  35 ,  some  evidence  is  found  to 
reject  the  null  hypothesis  of  no  change  in  ALOS  from  1976  to  1977. 
For  seven  of  the  27  diagnoses  considered  in  the  analysis,  a  statis- 
tically significant  decrease  in  ALOS  was  observed  for  Stage  1 
patients,  with  no  significant  increases.  For  twenty  diagnoses  there 
was  no  change. 

For  Stage  2/3  patients,  the  results  were  similar,  although  not 
quite  as  consistent.  For  six  diagnoses,  significant  decreases  in 
mean  LOS  were  pbserved;  for  two  diagnoses,  marginally  signifi- 
cant  increases   were   observed.      Nineteen   diagnoses  had  no  change. 

Turning  to  the  second  null  hypothesis,  there  appears  to 
be  sufficient  evidence  to  reject  the  hypothesis  that  there  have 
been  no  differences  in  ALOS  change  between  Stage  1  and  Stage  2/3 
patients.  There  is  no  clear  pattern,  however,  to  the  direction  of 
the  change;  PSRO  review  appeared  to  be  more  effective  (in  the 
sense  of  reduced  ALOS)  for  Stage  2/3  patients  for  five  of  the 
diagnoses  studied  and  more  effective  for  Stage  1  patients  for  three. 
There  was  no  difference  for  nineteen  of  the  diagnoses. 
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2.5.4.4    Relationship  of  ALPS  and  Selected  PSRO  Area 


Characteristic  Variables 


Additional  variables  were  selected  for  further  analysis 
to  investigate  whether  or  not  the  understanding  of  the  relationship 
between  ALOS,  severity  of  illness,  and  time  could  be  improved  by 
controlling  for  PSRO-specif ic  supply  characteristics.  The  variables 
selected  for  this  analysis  include: 


•  Short-term  beds  per  1,000  area  population. 

•  MDs  per  1,000  PSRO  area  population. 

•  Extended  care  beds  per  1,000  PSRO  area  populations 
aged  65  and  over. 


As  stated  earlier,  each  of  these  variables  was  dichotomized: 
"high"  vs.  "low".  Separate  regression  analyses  were  performed 
to  test  the  significance  of  these  variables  in  explaining  the 
relationship  between  ALOS,  severity  of  illness  (Stage  1  vs.  Stage 
2/3)  and  time  (1976  vs.  1977).  The  regression  took  the  form 
discussed  in  the  Section  2.5.3.2. 

Again,  two  null  hypotheses  were  tested  for  each  of  the  re- 
sources in  question: 

HQ:  The  change  in  stage-specific  ALOS  from  1976  to  1977  is 
independent  of  whether  the  PSRO  area  is  characterized  by  a 
high  or  low  supply  of  resources. 

H^:  There  has  been  no  difference  between  PSROs  of  different 
supply  characteristics  in  changes  in  ALOS  as  a  function 
of  stage  of  illness. 

Differences  in  disease-specific  ALOS  over  the  early/late  period 
can,  to  some  extent,  be  attributed  to  the  availability  of  resources 
within  the  PSRO.  Of  the  resources  analyzed,  extended  care  beds 
per  1  ,000  population  aged  65  and  over  appears  to  be  the  most  sig- 
nificant. For  this  characteristic,  statistically  significant 
differences  in  ALOS  for  Stage  1  patients  were  found  for  three 
diagnoses:  bronchial  asthma,  cholecystitis  and  hypertension. 
Although  no  difference  in  LOS  had  been  found  for  these  diagnoses 
in  the  initial  analysis,  when  controlling  for  the  area  supply 
characteristics,  a  statistically  significant  difference  in  ALOS 
does  emerge.  Specifically,  differences  in  the  ALOS  for  Stage  1 
hypertension    and    cholecystitis    patients    decreased    more    in  high 
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and  low  supply  areas,  respectively.  For  those  remaining  diagnoses 
where  differences  in  ALOS  were  found  in  the  initial  analysis,  no 
additional  effects  were  found  for  area  supply  characteristics, 
indicating  that  changes  in  utilization  patterns  for  these  diagnoses 
were  not  related  to  the  availability  of  extended  care  facility  beds 
in  the  PSRO  area. 

The  next  area  resource  characteristic  examined  was  the  number 
of  non-Federal  active  MDs  per  1,000  population.  The  results  reveal 
that  a  greater  density  of  physicians  results  in  longer  lengths  of 
stay  (i.e.,  smaller  differences  in  ALOS).  Significant  differences 
among  PSRO  types  in  Stage  1  ALOS  were  observed  for  2  diagnoses: 
bacterial  pneumonia  and  diabetes  mellitus.  In  addition,  differences 
in  ALOS  for  Stage  2/3  patients  between  high  and  low  MDs  per  1,000 
were  found  for  seven  diagnoses.  The  regression  model  indicates 
decreases  in  ALOS  were  greater  for  the  low  supply  areas  for  benign 
prostatic  hypertrophy,  diabetes  mellitus,  diverticulitis  and 
hypertension.  This  supports  the  notion  of  longer  lengths  of  stay  in 
areas  with  high  ratios  of  MDs  per  1,000  population. 

Finally,  short-term  hospital  beds  per  1,000  PSRO  population 
were  relatively  insignificant  in  explaining  differences  in  disease- 
specific  changes  in  ALOS  from  1976  to  1977. 

This  expanded  regression  approach  tries  to  determine  which 
resource  characteristics  of  the  PSRO  are  associated  with  improved 
utilization  of  the  hospital.  The  results  suggest  that  the  ability 
of  PSROs  to  affect  utilization  is  constrained  by  the  resource 
characteristics  of  the  areas  in  which  the  PSROs  operate. 

2.5.5    Medicaid  Utilization  Analysis 

In  this  section  the  analysis  and  findings  related  to  changes  in 
hospital  utilization  patterns  for  Medicaid  recipients  are  presented. 
As  with  the  Medicare  analysis,  patients  were  assigned  to  stages 
based  on  the  diagnostic  codes  used  in  the  PHDDS  data  system.  An 
arbitrary  minimum  of  100  patients  per  diagnosis  was  established. 
After  dropping  from  consideration  the  diagnoses  that  did  not  meet 
this  criterion,  twenty-one  remaining  diagnoses,  representing  23,849 
patients,  were  used  for  the  analysis.    These  are  listed  in  Table  36. 


135 


TABLE  36 


DIAGNOSES  USED  IN  STAGING  ANALYSES  OF  MEDICAID  PATIENTS 


Diagnosis 

Alcoholism 

Appendicitis 

Bacterial  Pneumonia 

Bronchial  Asthma 

Bronchopneumonia 

Carcinoma  of  Cervix 

Cholecystitis 

Diabetes  Melitus 

Diarrhea  Gastritis 

Dysfunctional  Uterine  Bleeding 

External  Hernia 

Hypertension 

Myocardial  Infarction 

Pancreatitis 

Pelvic  Inflammation 

Peptic  Ulcer 

Pneumonia,  Unspecified 

Renal  Calculus  Disease 

Spontaneous  Abortion 

Thrombophlebitis 

Urinary  Tract  Infection 

2.5.5.1    Analysis  of  Changes  in  Diagnostic  Case  Mix 

The  following  null  hyphothesis  was  tested  using  a  Chi-Square 
test  for  each  of  the  twenty-one  diagnoses  included  in  the  analysis: 

H  :  There   has   been   no   change   in  the  proportion   of  patients 
by  stage  from  1976  to  1977. 

A  pattern  of  a  shift  toward  a  more  complex  case  mix  from  1976 
to  1977  was  observed  for  three  diagnoses:  peptic  ulcer  (significant 
at  the  .10  level)  and  hypertension  and  urinary  tract  infection 
(statistically  significant  at  the  .05  level).  In  no  case  was  a 
statistically  significant  shift  found  in  the  direction  of  a  less 
complex  case  mix.  Eighteen  diagnoses  did  not  shift  in  either  direc- 
tion. 

While  the  number  of  diagnoses  for  which  there  were  shifts  in 
stage  distribution  was  less  for  Medicaid  patients  than  for  Medicare 
patients,  the  pattern  of  a  move  toward  a  more  complex  case  mix  is 
again  observed. 
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2.5.5.2    Length  of  Stay  Analysis 


Applying  the  regression  model  to  the  Medicaid  data,  observed 
changes  in  mean  length  of  stay  were  analyzed.  Again,  two  null 
hypotheses  were  tested: 

H  :  There  has  been  no  change  in  stage-specific  ALOS  from  1976 
to  1977. 

H^:  There   have   been   no   differences    in   changes    in   ALOS   as  a 
function  of  stage  of  illness. 

The  results  of  the  analysis  are  displayed  in  Table  37. 

Reviewing  the  results  in  Table  37,  a  statistically  significant 
decrease  in  ALOS  for  Stage  1  patients  was  observed  for  three  of  the 
diagnoses  included  in  the  analysis;  none  of  the  diagnoses  showed 
significant  increases  while  for  eighteen  diagnoses  there  was  no 
shift  in  either  direction.  For  Stage  2/3  patients,  the  results  were 
similar.  For  four  diagnoses,  significant  decreases  in  mean  LOS  were 
observed  while  for  one  (pancreatitis)  there  was  an  increase. 
Sixteen  diagnoses  showed  no  significant  differences. 

Turning  to  the  second  null  hypothesis,  there  appears  to 
be  some  evidence  to  reject  the  premise  of  no  differences  in  ALOS 
change  between  Stage  1  and  Stage  2/3  Meidcaid  patients.  For  four 
diagnoses,  significantly  greater  decreases  in  ALOS  were  observed  for 
Stage  2/3  patients  than  for  Stage  1  patients.  For  pancreatitis,  the 
increased  mean  LOS  for  Stage  2/3  patients  was  significantly  dif- 
ferent from  the  change  for  Stage  1  patients. 

The  differential  impacts  for  Medicaid  are  consistent  with  the 
findings  of  the  Medicare  analysis. 

2.5.6  Assessment  of  the  Comparability  Between  PHDDS 
and  Abstract  Service  Hospital  Discharge  Data : 
Medicare,  January-September,  1976 

2.5.6.1  Objectives 

The  purpose  of  this  assessment  is  to  evaluate  the  comparability 
of  automated  hospital  discharge  data  abstracted  by  two  different 
organizations  —  a  professional  abstract  service/ 1  and  the  local 
PSRO.  The  assessment  is  preliminary  to  a  planned  pre-PSRO/  post- 
PSRO  stage-specific  utilization  analysis,  using  abstract  data  for 
the  pre-PSRO  and  PHDDS  data  for  the  post-PSRO  period.  A  necessary 
prerequisite  for  a  comparative  analysis  using  two  different  data 
bases  is  that  the  data  bases  themselves  are  comparable.  The  compar- 
ability was  tested  by  applying  the  staging  methodology  to  both  data 
bases  covering  the  same  hospitals  for  the  same  time  period. 

/1       The  Commission  on  Professional  and  Hospital  Activities  (CPHA) 
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2.5.6.2  Results 


A.      Distribution  of  Patients  by  Stage 

For  six  of  the  twenty-five  diagnoses,  significantly  higher 
levels  of  severity  were  observed  for  the  CPHA  than  for  the  PHDDS 
discharges  (alcoholism,  benign  prostatic  hypertrophy,  carcinoma  of 
the  breast,    colon   carcinoma,   diabetes  mellitus  and  hypertension)* 

An  example  of  the  Chi-Square  test  performed  on  colon  carcinoma 
is  shown  in  Table  38. 

For  CPHA,  almost  55%  of  the  1976  colon  carcinoma  patients 
were  in  Stage  1  while  for  PHDDS,  over  78%  of  the  colon  carcinoma 
patients  were  in  Stage  1.  This  statistically  significant  differ- 
ence, coupled  with  an  almost  perfect  match  for  total  number  of 
patients  (121  for  CHPA  and  123  for  PHDDS),  suggests  the  possibility 
that  some  discharges  may  have  been  miscategorized  in  a  lower  stage* 


TABLE  38 

MEDICARE  DISTRIBUTION  OF  PATIENTS  BY  STAGE  AND  ABSTRACT 
TYPE:     STAGED  DIAGNOSIS  -  COLON  CARCINOMA 


ABSTRACT  TYPE 


STAGE 

CPHA 

PHDDS 

TOTAL 

Stage  1 

66 
(54.44%) 

96 
(78.05%) 

162 
(66.39%) 

Stages 
2  &  3 

55 
(45.45%) 

27 

(21.95%) 

82 

(33.61%) 

Total 

121 

( 49 . 59% ) 

123 
(50.41%) 

244 
( 100.00%) 

Statistics  for  2-way  tables 
Chi-Square  =  15.101 

Degrees  of  freedom    =  1 
PROB  =  0.0001 
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It  is  also  possible  since  the  CP HA  data  shows  a  consistently  higher 
severity  level  that  key  secondary  diagnoses  qualifying  as  compli- 
cations of  the  staged  conditions  may  not  have  been  recorded  on  the 
PHDDS  abstracts.  Verification  of  this  preliminary  conclusion/ 
however,  is  beyond  the  scope  of  the  present  analysis* 

The  implication  of  the  above  finding  of  significant  differ- 
ences between  the  distribution  of  Medicare  patients  by  stage  for 
CP  HA  vs.  PHDDS  1976  data,  is  that  the  planned  pre-PSRO/post-PSRO 
impact  analysis  using  CP HA  data  for  the  pre,  and  PHDDS  data  for  the 
post,  would  not  be  valid  at  the  present  time. 

B.      ALPS  Comparison 

In  addition  to  analysis  of  the  distribution  of  patients 
by  stage,  a  comparison  of  ALOS  for  PHDDS  and  CP  HA  abstracts  was 
conducted. 

A  number  of  patterns  merit  comment.  First,  the  difference  in 
ALOS  for  Stage  1  alcoholism  is  7.9  days  for  CPHA  and  12*5  days 
for  the  PHDDS  data.  This  finding  is  consistent  with  the  significant 
difference  in  stage  distribution  for  alcoholism  noted  above.  While 
the  total  number  of  patients  (83  from  CPHA,  95  from  PHDDS)  is 
similar,  only  8%  of  the  CPHA  patients  were  categorized  as  Stage  1 
while  59%  of  the  PHDDS  patients  were  categorized  as  Stage  1.  If  one 
assumes  that  these  are  the  same  patients  and  that  PHDDS  is  not 
recording  all  complications  for  these  patients,  a  possible  explana- 
tion for  the  higher  LOS  for  Stage  1  alcoholism  for  PHDDS  is  that 
many  of  these  patients  are  actually  miscategorized  Stage  2/3 
patients. 

For  other  diagnoses,  no  clear  explanation  for  the  discrep- 
ancies is  apparent. 

In  summary,  it  is  clear  that  significant  differences  exist 
between  the  PHDDS  and  CPHA  abstracts.  While  it  appears  from  a 
limited  statistical  analysis  that  the  CPHA  abstracts  include  a 
more  complete  record  of  diagnoses  and  complications,  a  much  more 
thorough  review  would  be  necessary  to  determine  the  cause (s)  of 
the  discrepancies.  Nevertheless,  the  results  suggest  that  PHDDS 
may  be  missing  important  information  crucial  to  staging  analysis* 
The  effect  of  missing  secondary  diagnoses  in  PHDDS  would  be  to 
underestimate  the  level  of  severity  of  staged  diagnoses.  This  does 
not  greatly  affect  the  primary  analysis  in  this  study,  however.  If 
the  staging  is  biased  towards  lower  severity,  the  bias  exists  in 
both  1976  and  1977  and  the  differences  between  the  two  years  still 
reflect  real  differences  (subject  to  the  results  of  the  abstracting 
study).  Improvements  in  the  PHDDS,  to  capture  more  of  the  informa- 
tion necessary  for  staging,  would  improve  the  efficiency  and  accu- 
racy of  future  studies  of  case  mix  severity. 
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2.5.7  Conclusions 


In  this  study,  the  staging  methodology  has  been  applied  to 
the  PSRO  Hospital  Discharge  Data  Set  (PHDDS).  The  results  of 
this  study  show  that  staging  is  potentially  a  reasonable  and 
useful  technique  for  studying  hospital  utilization. /1 

Analyses  of  case  mix  and  length  of  stay  in  selected  PSRO 
areas  were  performed  separately  for  the  Medicare  and  Medicaid 
patient  populations. 

The  following  was  demonstrated  for  Medicare  patients: 

•  The  case  mix,  as  measured  by  the  stage  distribution, 
of  patients  treated  in  the  studied  PSROs  has  changed. 
Quantitative  evidence  was  found  of  shifts  to  higher 
levels  of  severity  for  those  types  of  diagnoses  gen- 
erally associated  with  hospitalization  being  required  only 
at  later  stages  of  illness  (e.g.,  hypertension).  In 
addition,  the  one  shift  to  a  lower  level  of  severity  was 
for  a  condition  in  which  early  detection  and  treatment  are 
essential,  carcinoma  of  the  breast.  If  these  shifts  in 
case  mix  are,  in  fact,  real  and  not  changes  in  medical 
record-keeping,  then  PSRO  review  may  indeed  be  associated 
with   instituting  a  more  appropriate  use  of  the  hospital. 

•  An  analysis  of  average  length  of  stay  (ALOS)  by  diagnosis 
revealed  significant  reduction  in  utilization  for  both 
Stage  1  and  Stage  2/3  patients  for  selected  diagnoses.  The 
decline  in  ALOS  for  Stage  1  patients  was  noted  for  those 
diagnoses  typically  not  requiring  hospitalization  at  Stage 
1.  One  interpretation  of  this  finding  is  that  review  has 
been  effective  in  limiting  the  unnecessary  use  of  hospital 
resources. 

The  following  was  demonstrated  for  Medicaid  patients: 

•  The  case  mix  of  Medicaid  patients  treated  in  the  studied 
PSROs  also  shifted  towards  a  more  complex  case  mix,  al- 
though for  a  smaller  number  of  diagnoses  than  the  Medicare 
population. 

•  In  the  ALOS  analysis  of  Medicaid  patients,  significant 
declines  in  ALOS  occurred  for  only  a  few  diagnoses  for 
both  Stage  1  and  Stage  2/3  patients. 


/ 1      For  a  previous  demonstration  of  the  staging  methodology 

applied   to   PHDDS,    see    A.    Stonecipher,    C.    Spirka,    J.  MCord, 
"Some  Preliminary  Results  from  Staging  Analyses  of  the 
PSRO  Hospital  Discharge  Data  Set,"  June  1978. 
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Finally/  the  compatabi li ty  between  PHDDS  and  abstract  service 
hospital  discharge  data  was  assessed.  It  was  possible  to  match 
patient  records  in  the  two  data  bases,  a  prerequisite  for  performing 
longitudinal  analysis  of  the  PSRO  review  program  on  a  diagnosis/pro- 
cedure-specific basis  at  the  present  time.  On  the  other  hand, 
for  the  time  period  common  to  both  data  bases  (January- September , 
1976),  significantly  higher  severity  of  illness  levels  were  found 
for  the  abstract  service  than  for  the  PHDDS  data.  This  finding 
appears  to  be  related  to  a  more  complete  recording  of  diagnoses  and 
complications  on  the  CP HA  abstract.  Due  to  those  discrepancies,  the 
contemplated  analysis  using  CPHA  data  for  the  pre-PSRO  period  and 
PHDDS   data   for   the  post-period  was   not  performed  at   this  time. 
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3.       PROGRAM  COST 


3.1  Background 

3.1.1  Objectives 

Although  there  are  many  possible  perspectives  on  the  area  of 
program  costs,  only  a  handful  of  these  were  amenable  to  analysis 
for  this  year's  evaluation.  Total  costs  and  unit  costs  for  the 
nation  for  1977  were  estimated  as  an  initial  objective.  The 
remaining  analyses  involve  examinations  of  variations  in  program 
costs  by  hospital,  PSRO,  or  PSRO  function.  Finally,  these  costs  are 
compared  to  the  FY  1979  national  unit  cost  objective  of  $8.70  to 
gain  an  initial  assessment  of  the  possible  difficulties  in  achieving 
sufficient  reductions  in  program  expenditures. 

3.1.2  Studies  and  Relationships 

Several  distinct  studies  are  reported.  One  study  involved 
a  descriptive  analysis  of  the  ranges  in  PSRO  program  expenditures  as 
reported  to  HSQB.  A  second  study  involved  a  detailed  examination  of 
systematic  variations  in  costs  incurred  in  delegated  hospitals. 
Byproducts  of  the  two  efforts  are  a  set  of  national  projections  of 
PSRO  expenditures  and  an  estimate  of  the  expenditures  for  concurrent 
review  for  Medicare  in  1977.  The  latter  estimates  were  used  in  the 
benefit-cost  analysis  reported  in  Section  4. 

It  is  important  to  note  that  these  figures  are  based  on 
the  costs  of  100%  review.  Few  PSROs,  if  any,  were  using  focused 
review  at  the  time  the  available  data  were  generated.  The  figures 
presented  represent  a  historical  picture  prior  to  what  are  expected 
to  be  major  administrative  changes.  These  analyses  should  be 
useful,  nonetheless,  because  they  should  give  an  indication  of  the 
magnitude  of  effort  necessary  to  reduce  program  expenditures  to  the 
level  desired  for  FY  1979.  In  addition,  the  studies  of  variations 
in  costs  may  provide  insights  into  areas  where  further  cost  reduc- 
tions might  be  achieved. 

3.1.3  Data  Sources 

The  costs  exclusive  of  delegated  hospitals  are  assessed 
using  data  from  HSQB  Form  151.  These  data  were  used  for  both  the 
descriptive  analysis  and  the  national  projections. 

The  data  sources  for  the  hospital  variation  study  include  a 
sample  of  1100  edited  HSQB  153  Forms,  the  delegated  hospital 
cost  reports.  A  file  on  hospital  characteristics  compiled  for  the 
1977    PSRO   evaluation   was    used   to   supplement    this    analysis.  The 
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results  of  this  analysis  were  merged  with  the  former  data  set  to 
project  national  program  costs. 

3*1.4    Relation  to  1977  Evaluation 

All  of  these  studies  are  continuations  or  refinements  of 
studies  undertaken  for  the  1977  PSRO  Evaluation.  The  differences 
from  last  year's  studies  are  as  follows: 

(a)  The  PSRO  Management  Information  System  (PMIS)  was  used  for 
the  first  time  to  develop  national  cost  estimates.  In  the 
past,  only  projections  involving  a  small  sample  of  PSROs 
were  possible. 

(b)  The  cost  estimates  produced  for  the  benefit-cost  com- 
parisons are  based  on  full  costs  rather  than  the  incre- 
mental costs  compared  to  the  old  system  of  utilization 
review.  Since  that  system  can  no  longer  be  considered  an 
alternative  to  PSRO,  the  full  cost  estimates  are  more 
appropriate . 

(c)  Only  operating  costs  are  computed  for  the  analysis  since 
startup  costs  are  considered  fixed  costs  which  have 
already  been  invested. 

(d)  Monitoring  expenditures  by  fiscal  intermediaries  and 
state  agencies  were   not  assessed  in  this  year's  report. 

3.2    Cost  Definitions 

There  are  a  variety  of  ways  that  program  expenditures  may  be 
partitioned  to  examine  costs.  In  addition,  in  the  course  of 
the  analyses,  different  data  sets  will  be  used  to  estimate  the  costs 
of  the  various  parts  of  the  program.  As  a  result,  there  are  slight 
differences  in  some  cost  estimates.  To  lessen  the  possible  con- 
fusion, a  series  of  definitions  are  presented  which  should  be  used 
consistently  for  the  cost  and  benefit-cost  analyses. 

One  way  of  partitioning  program  expenditures  is  by  source 
of  funding  and  by  the  funding  mechanism.  PSRO  funds  come  from 
either  general  revenue  appropriations  to  HCFA  or  through  the  Social 
Security  Trust  Funds.  Funds  are  distributed  either  through  grants 
or  contracts  directly  to  the  PSROs  or  through  payments  to  hopitals 
through  fiscal  intermediaries.  Table  39  illustrates  the  four-part 
categorization  of  expenditures  developed  by  HSQB. 
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Table  39:     Program  Costs  Partitioned  by  Funding  Source 


Cost  Category 

Funding  Source 

Funding  Mechanism 

Part 

I 

Management  and  sup- 
port and  non-acute 
care  review 

HCFA-appropriated  PSRO  contract/grant 
funds 

Part 

II 

Nondelegated  review 

Trust  Funds 

PSRO  contract/grant 

Part 

III 

Areawide  hospital 
review 

Trust  Funds 

PSRO  contract/grant 

Part 

IV 

Delegated  review 

Trust  Funds 

Funding  directly  to 
institution  through 
fiscal  intermediary 

Costs  can  also  be  partitioned  by  program  function.  There 
are  two  functions  which  fall  entirely  within  Part  I  costs: 

( 1 )  program  management  and  support;  and 

(2)  non-acute  care  review  (long-term  care  review  and  ambula- 
tory care  review) . 

The  remaining  functions  taken  together  have  been  labelled 
"Total  Hospital  Review".  The  three  functions  under  this  category 
are  areawide  review  (profiles,  data  processing,  criteria  develop- 
ment, hospital  assessment,  and  training  and  clerical  support  for 
those  functions),  concurrent  review  (admission  certification  and 
continued  stay  review),  and  medical  care  evaluation  (MCE)  studies. 

Analyses  of  variations  in  cost  by  PSRO  or  by  function  require 
an  adjustment  to  take  into  account  the  differences  in  the  number  of 
patients  affected  by  the  various  parts  of  the  program.  For  this 
reason,  unit  cost  estimates  are  provided.  These  are  computed  by 
dividing  annual  expenditures  by  the  relevant  number  of  hospital 
discharges  which  were  subject  to  review. 

The  difference  between  "discharges  subject  to  review"  and 
"discharges  reviewed"  will  become  more  important  as  PSROs  adopt 
focusing.  However,  for  the  1977  cost  estimates  in  this  report,  unit 
costs  per  discharge  subject  to  review  and  unit  costs  per  discharge 
reviewed  are  virtually  identical,  since  virtually  all  discharges 
subject  to  review  were  reviewed. 

For  the  benefit-cost  analysis  it  was  necessary  to  develop 
cost  estimates  for  PSRO  activities  most  related  to  utilization 
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control.  This  involved  combining  all  of  the  costs  for  concurrent 
review,  some  of  the  costs  of  areawide  review,  and  such  management 
and  support  costs  as  could  be  allocated  to  those  functions. 

Management  and  support  costs  were  allocated  to  each  function 
using  the  total  direct  expenditures  for  each  function  as  a  percen- 
tage of  all  expenditures  for  review  (including  non-acute  care 
review).  Further,  areawide  review  costs  were  allocated  to  concur- 
rent review  using  the  ratio  of  concurrent  review  expenditures  to  MCE 
expenditures.  The  resulting  estimates  should  be  considered  "total 
concurrent  review." 

Finally,  a  distinction  should  be  made  between  total  hospital 
review  and  what  will  be  called  "total  PSRO  review."  The  former  does 
not  include  any  Part  I  funds.  The  latter  combines  funds  from  all 
funding  sources,  Parts  I,  II,  III,  and  IV. 

3.3    Budget  Analyses  of  Unit  Costs 

3.3.1  Objectives 

The  primary  objective  of  this  analysis  was  to  examine  the 
range  of  costs  for  various  PSRO  activities.  A  second  objective  was 
to  identify  any  systematic  differences , in  costs  between  PSROs  which 
were  already  fully  implemented  and  those  which  experienced  substan- 
tial increases  in  implementation. 

3.3.2  Data 

Data  for  this  analysis  came  from  the  PMIS.  A  unit  cost  report 
for  FY  1977  by  PSRO  area,  prepared  by  HSQB,  was  the  primary  source 
document.  This  was  supplemented  by  quarterly  cost  reports,  also 
from  the  PMIS. 

PSROs  were  excluded  from  the  analysis  if  they  had  not  been 
in  operation  for  at  least  three  quarters  with  at  least  fifty  dis- 
charges in  each  quarter.  PSROs  with  only  incomplete  data  also  were 
excluded.  As  a  result,  seventy-six  PSROs  were  included  in  this 
analysis. 

3.3.3  Methods 

Variations  in  unit  costs  across  HEW  regions  are  presented 
in  terms  of  the  weighted  averages  in  each  region.  That  is,  combined 
expenditures  for  the  various  functions  are  divided  by  the  total 
discharges  subject  to  review  in  the  region.  Variations  in  unit 
costs  across  PSROs  are  assessed  in  terms  of  ranges  and  median  unit 
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costs  for  the  various  functions.  In  addition,  some  simple  cross- 
tabulations  were  prepared  to  identify  any  consistent  differences  in 
costs  between  fully  implemented  PSROs  and  those  which  experienced 
substantial  increases  in  implementation  over  FY  1977. 

3.3.4  Findings 

There  was  considerable  variation  in  unit  costs  across  the 
various  HEW  regions.  HEW  Region  IX  had  the  highest  unit  cost  for 
total  hospital  review,  areawide  review,  and  concurrent  review  at 
$15.30,  $3.09,  and  $10.85,  respectively.  Region  X  had  the  highest 
average  unit  MCE  cost  at  $2.30.  The  lowest  average  unit  costs  by 
region  were  as  follows:  areawide — $1.12  in  Region  V;  concurrent 
review — $6.68  in  Region  VI;  and  MCE — $1.00  in  Region  IV.  Region  VII 
had  the  lowest  average  unit  cost  for  total  hospital  review — $9.87. 
These  results  are  presented  in  Table  40 . 


TABLE  40 

HOSPITAL  REVIEW  UNIT  COST  SUMMARY 
FY  1977 


  | 

Average  Unit  Cost  in  Dollars 

Total 

Hospital 

Areawide 

Concurrent 

Region 

Review/ 1 

Review 

Review 

MCE 

I 

$12.47 

$2.20 

$  9.16 

$1.11 

II 

14.40 

2.84 

10.13 

1.43 

III 

14.42 

2.20 

10.66 

1.56 

IV 

11.30 

1.74 

8.56 

1 .00 

V 

9.92 

1.12 

7.32 

1.48 

VI 

10.03 

1 .73 

6.68 

1.62 

VII 

9.87 

1 .76 

6.94 

1.17 

VIII 

12.55 

1.13 

9.73 

1.69 

IX 

15.30 

3.09 

10.85 

1.36 

X 

1 1 .26 

1.69 

7.27 

2.30 

National/2 

12.08 

1.94 

8.78 

1.36 

/1     Management  support  costs  are  excluded. 

/2  National  average  figures  were  computed  by  dividing  the  total 
costs  for  each  function  by  the  total  discharges  related  to 
those  specific  costs. 


147 


The  national  average  unit  cost  for  total  hospital  review 
for  conditional  PSROs  which  were  performing  review  at  the  end  of  FY 
1977  was  $12.08.  The  range  went  from  a  low  of  $6.52  in  9CA08  /1_ 
to  a  high  of  $74.90  in  9CA23.  The  latter  area,  however,  was  only  33 
percent  implemented  by  the  end  of  FY  77.  In  fact,  relatively  new 
conditional  PSROs  most  often  reported  very  high  unit  costs.  This 
was  usually  due  to  several  months  of  expenditures  prior  to  the 
initiation  of  binding  review. 

The  distribution  of  unit  costs  for  total  hospital  review 
is  displayed  in  Table  41.  This  table  highlights  the  relatively  high 
unit  costs  of  less  than  fully  implemented  conditional  PSROs.  Only 
seven  of  the  forty-two  PSROs  which  reviewed  in  excess  of  90  percent 
of  their  Medicare  discharges  in  calendar  1977  had  unit  costs  of 
total  hospital  review  above  $15.  In  contrast,  over  one-third  of 
those  PSROs  which  reviewed  between  40  to  90  percent  of  their  Medi- 
care discharges  had  unit  costs  above  $15,  and  seven  of  the  ten  PSROs 
which  were  less  than  forty  percent  implemented  had  unit  costs 
greater  than  $20. 

Table  41  presents  a  mixed  picture  with  respect  to  the  like- 
lihood of  achieving  the  FY  1979  unit  cost  target  of  $8.70  for 
total  hospital  review. /2  Thirteen  PSROs  reported  FY  1977  unit 
costs  of  $9.00  or  lower.  (Nine  were  below  $8.70.)  Further,  the 
mode  of  the  distribution  for  fully  implemented  PSROs  was  in  the  $9 
to  $10  range.  Hence,  there  was  a  significant  number  of  PSROs  within 
one  or  two  dollars  of  the  cost  target. 

More  often  than  not,  however,  unit  costs  were  reported  in 
excess  of  $10.  Despite  the  observed  peak  between  $9  and  $10, 
the  distribution  of  unit  costs  was  fairly  uniform  up  to  $15.  The 
median  unit  cost  reported  was  $12.31  in  9CA27,  and  the  median  for 
those  PSROs  which  were  at  least  ninety  percent  implemented  was 
$11.60  in  9CA04.  Reductions  in  units  costs  should  come  about 
through  increases  in  implementation.  Focusing  should  enable  still 
lower  unit  costs  on  the  average,  but  in  some  PSROs  the  reductions 
needed  to  meet  the  FY  1979  objective  would  not  be  trivial. 

Total  PSRO  review  also  showed  considerable  variation.  The 
low  for  FY  1977  was  $9.70  in  6AR01.  The  high  was  $104.74  in  1VT01, 
an   area   which  was   33  percent   implemented  at  the  end  of   FY  1977. 


/1  HSQB  codes  will  be  used  to  identify  individual  PSROs.  The  code 
for  each  PSRO  consists  of  the  HEW  region  number,  the  two  letter 
state  abbreviation,  and  the  PSRO  number  within  the  state. 

/2  The  unit  cost  target  of  $8.70  for  total  hospital  review  is  an 
HSQB  objective  designed  for  compatibility  with  the  overall 
program  budget. 
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TABLE  41:     TOTAL  HOSPITAL  REVIEW  UNIT  COSTS,   FY  1977 


NUMBER  OF  PSROs  REPORTING  UNIT  COSTS 
WITHIN  COST  RANGE  SHOWN 

ALL  PSROs 
COMBINED 

Breakdown  By  Percentage  of  Medicare 
Discharges  Reviewed/ 1 

Cost 

ReUlCJS 

90% 
Or  More 

40-90% 

Less  than 
40% 

$6-$8 

5 

4 

1 

0 

8 

4 

4 

0 

9i»   9  1 U 

7 

7 

0 

0 

6 1 n_< 1 1 
9 1 U— 9 1 1 

7 

3 

3 

1 

$11-$12 

8 

5 

3 

0 

$12-$13 

5 

4 

1 

0 

$13-$14 

8 

4 

3 

1. 

$14-$15 

5 

4 

0 

1 

$15-$20 

5 

4 

0 

$20-$25 

7 

2 

3 

2 

$25+ 

7 

0 

2 

5 

TOTAL 

76 

42 

24 

10 

/1    Percentage  of  Medicare  discharges  reviewed  is  for  calendar, 
not  fiscal,  1977. 
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The  weighted  average  cost  for  total  PSRO  review  was  $17.71  for  the 
seventy-six  PSROs  with  complete  data.  The  1977  PSRO  evaluation 
estimated  a  cost  of  total  PSRO  review  of  $17.32  in  1976;  however, 
this  estimate  did  not  include  any  startup  costs.  In  order  to  get  a 
comparable  estimate  using  FY  1977  data,  PSROs  which  did  not  review 
at  least  60  percent  of  their  Medicare  discharges  in  CY  1977  were 
excluded  from  the  analysis.  The  FY  1977  unit  cost  of  total  PSRO 
review  for  the  remaining  PSROs  was  $15.60. 

The  distribution  of  unit  costs  for  concurrent  review  is 
presented  in  Table  42.  Regardless  of  whether  delegated  and  non- 
delegated  costs  are  taken  separately  or  combined,  the  modal  unit 
cost  is  in  the  $6  to  $8  range.  This  is  close  to  the  unit  concurrent 
review  cost  needed  to  achieve  the  FY  1979  target  cost  for  total 
hospital  review. 

The  ranges  of  unit  costs  are  reported  in  Table  43.  Again,  unit 
costs  are  lower  when  one  examines  only  PSROs  which  had  reviewed  more 
than  90  percent  of  their  Medicare  discharges  in  CY  1977.  This  can 
be  seen  in  comparisons  of  both  the  maxima  and  the  medians  of 
distributions.  In  most  instances,  the  maximum  reported  unit  cost 
comes  from  a  PSRO  area  with  a  low  implementation.  (PSRO  ORO02  had  a 
very  high  unit  cost  -  $81.13  -  for  nondelegated  concurrent  review, 
but  only  sixteen  discharges  were  reviewed. ) 

MCEs  exhibited  an  over  one-hundred  fold  variation  ($.08  to 
$9.19)  compared  to  a  less  than  sixfold  variation  for  concurrent 
review  ($3.25  to  $18.09).  This  is  due  to  the  variations  in  hospital 
sizes  across  the  PSROs  which  lead  to  variations  in  the  numbers  of 
required  MCEs.  MCEs  are  usually  implemented  following  implemen- 
tation of  concurrent  review,  which  would  tend  to  increase  the 
observed  variation  in  1977. 

Finally,  there  is  some  evidence  of  possible  economies  of 
scale  not  directly  related  to  variations  in  implementation  status. 
This  evidence  is  presented  in  Table  44.  Median  unit  costs  are  seen 
to  be  lower,  the  greater  the  number  of  discharges  reviewed.  Al- 
though this  findings  is  hardly  definitive,  it  suggests  that  there 
may  be  some  savings  in  unit  cost  which  would  occur  if  PSROs  extended 
their  reviews  to  non- Federal  patient  populations. 
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TABLE  42:     CONCURRENT  REVIEW  UNIT  COSTS,   FY  1977 


NUMBER  OF  PSROs  REPORTING  UNIT  COSTS 
WITHIN  COST  RANGE  SHOWN 

Cost  Range 

All 
Hospitals 

Delegated 
Hospitals  Only 

Nonde legated 
Hospitals  Only 

$3-$6 

5 

8 

3 

$6-$8 

26 

22 

9 

$8-$ 10 

18 

19 

7 

$10-$12 

15 

13 

6 

6 

6 

4 

$14-$16 

3 

3 

3 

$16-$18 

2 

1 

4 

$18-$20 

1 

1 

1 

$20-$25 

0 

0 

4 

$25-$30 

0 

0 

2 

$30+ 

0 

0 

2 

TOTAL 

76 

73 

45 
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TABLE  43: 


RANGE  IN  UNIT  COST  VARIATION  FOR  TOTAL  PSRO  REVIEW 
BY  COMPONENT  OF  COST  AND  DELEGATION  STATUS 
FY  1977 


UNIT  COST  VARIATION 

Function 

All  PSROs 

90%  Implemented  PSROs 

Minimum 

Median 

Maximum 

Median 

Maximum 

Total  PSRO 
Review 

$9.70 
(6AR01) 

$19.56 
(3PA02 

$104.71 
( IVT0 1 ) 

$15.84 
(3PA07) 

$28.90 
(8WY01) 

Management  and 
Support 

1.11 

(6AR01) 

5.07 
(3PA02) 

81.44 
( IVT0 1 ) 

4.52 
(1CT01) 

14.83 
(3MD06) 

Total  Hospital 
Review 

6.52 
(9CA08) 

12.31 
(9CA27) 

74.90 
(9CA23) 

1 1 .66 

(9CA04) 

22.53 
(8WY01) 

Concurrent 
Review 

3.25 
( SMNO 1 ) 

8.76 
3MD03 

18.09 
(9CA24) 

7.76 
(9CA04) 

17.89 
(8WY01) 

MCE 

.08 
(9CA08) 

1.40 
(2NY05) 

9.19 
(9CA24) 

1.40 
(2NY05) 

4.86 
(9NV01) 

Areawide 
Review 

.08 
( 5WI0 1 ) 

2.21 
(8ND01) 

59.07 
(9CA23) 

1.43 
(50H02) 

6.16 
(2NY10) 

UNIT 

COST  VARIATION  BY  DELEGATION  STATUS 

Delegated  CR 

3.25 
( 5MN0 1 ) 

8.76 
(2NJ02) 

18.09 
(9CA024) 

8.76 
(1MA02) 

15.13 
(5WI01) 

Nonde legated  CR 

4.16 
(9NV01) 

10.94 
(1CT02) 

81.13 
(00R02) 

10.78 
(1MA02) 

24.03 
(5WI01) 

Delegated  MCE 

.20 
(9HI01) 

1.35 
I7M002) 

9.19 
(9CA24) 

1.35 
(7MO02) 

4.87 
(9NV01) 

Nonde legated  MCE 

.08 
(9CA08) 

1.64 
(8VT01) 

22.69 
(50402) 

1.54 
(5IL04) 

13.99 
(6NM01) 

PSRO  identifiers  are  in  parentheses. 
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TABLE  44 

Relationship  Between  PSRO  Median 
Unit  Costs  and  Review  Volume 


Median  Unit  Cost  of  Total  Hospital  Review 

Number  of  Discharges 
Reviewed 

90-100% 
Implementation 

40-90% 
Impl  erne  n  t  at  i  on 

0-24,999 

$13.89 

$19.07 

25,000  -  74,999 

11.41 

12.67 

75,000  + 

9.51 

13.48 

3.4    Analysis  of  Delegated  Hospital  Review 
Costs  for  Calendar  Year  1977 

3.4.1  Ob j  eqtives 

The  objectives  of  the  delegated  review  cost  analyses  are 
four-fold:  (a)  to  develop  a  representative  data  base  of  delegated 
hospitals  along  with  information  concerning  hospital  characteris- 
tics; (b)  to  analyze  delegated  hospital  costs  according  to  specific 
cost  categories;  (c)  to  identify  the  influence  of  hospital  charac- 
teristics on  delegated  review  costs;  and  (d)  to  suggest  cost  anal- 
ysis methods  for  ongoing  program  analysis  and  evaluation. 

3.4.2  Data 

Data  on  projected  review  costs  in  delegated  hospitals  came  from 
a  sample  of  approximately  1100  hospitals  for  which  edited  HSQB  153 
forms  were  available.  This  original  selection  of  hospitals  was 
not  entirely  representative  of  all  hospitals  in  the  country; 
therefore  HSQB  153  data  were  solicited  from  about  85  additional 
hospitals.  The  total  number  of  hospitals  in  the  final  sample  was 
1185. 

Cost  projections  from  the  forms  were  derived  from  the  process 
of  negotiating  reimbursement  rates  (from  fiscal  intermediaries)  for 
delegated  hospital  review.  For  ninety  of  the  1185  hospitals, 
however,  both  actual  and  projected  cost  data  were  available.  An 
examination  of  the  correspondence  between  these  data  was  performed 
to  assess  the  accuracy  of  the  estimates. 
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An  additional  source  of  potential  error  in  the  estimates  is  due 
to  the  fact  that  only  partial  year  data  was  available  from  some  of 
the  hospitals.  These  data  were  annualized  by  simple,  linear  extra- 
polation. This  might  tend  to  bias  upward  the  estimates  for  newly 
delegated    hospitals    working   with    a    cash-flow    accounting  system. 

An  additional  source  of  data  was  a  file  of  hospital  character- 
istics which  had  been  developed  in  the  course  of  the  1977  PSRO 
evaluation.  Data  from  this  file  were  linked  with  the  cost  estimates 
by  hospitals.  Hence,  241  variables  were  available  for  each  of  the 
1185  hospitals. 

3.4.3  Methods 

The  assessment  of  the  correspondence  between  the  projected  data 
and  the  actual  data  involved  estimation  of  correlation  coefficients 
using  data  by  expenditure  type  and  review  function.  For  example,  in 
the  89  hospitals  which  spent  at  least  $10  for  review  coordinators 
for  concurrent  review,  the  correlation  between  actual  and  projected 
cost  data  was  .926.  (This  was  statistically  significant  at  a  level 
of  5  percent.)  In  addition,  tests  were  employed  to  check  for 
statistically  significant  differences  in  the  average  annual  expen- 
diture estimates  derived  from  both  data  sources.  The  results  of 
these  tests  are  reported  in  Section  3.5.5. 

The  analyses  of  the  complete  file  involved  cross  tabulations  to 
identify  systematic  differences  in  unit  costs  according  to  hospital 
characteristics.  Additional  correlation  analyses  were  conducted  to 
identify  both  cost  characteristics  and  emerging  patterns  that  might 
be  employed  in  future  program  evaluation  and  monitoring. 

3.4.4  Limitations 

The  primary  limitation  in  the  analyses  is  the  uncertainty 
introduced  by  the  use  of  projected  cost  data.  Although  all  hos- 
pitals for  which  both  data  sources  were  available  were  included  in 
the  assessment  of  differences,  less  than  10  percent  of  the  hospitals 
(90  out  of  1185)  had  both  sources.  Further,  it  was  not  possible  to 
select  hospitals  on  a  random  basis,  so  systematic  biases  may  exist 
in  the  estimates. 

A  second  limitation  is  that  there  are  no  data  in  this  file 
on  the  costs  of  focused  review.  Again,  this  means  that  these 
analyses  can  only  present  a  picture  as  of  1977  of  the  interrelation- 
ships between  hospital  characteristics  and  the  costs  of  review. 
However,  it  is  believed  that  those  relationships  between  hospital 
characteristics  and  costs  do  provide  insights  that  will  be  useful  in 
monitoring  even  a  focused  review  system. 
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3.4.5  Findings 


The  analyses  of  the  correspondence  between  projected  data 
and  actual  data  identified  strong  positive  correlations  between  the 
two  data  sets  for  most  cost  categories.  Table  45  displays  these 
results.  In  only  two  of  seventeen  categories  were  the  correlations 
not  significantly  different  from  zero. 

In  four  cases,  however,  in  spite  of  a  significant  correlation 
there  was  also  a  significant  difference  in  the  estimated  means. 
Further,  the  averages  based  on  estimated  data  consistently  exceeded 
the  averages  based  on  actual  data.  The  average  projected  annual 
total  expenditure  exceeded  the  actual  average  by  more  than  ten 
percent . 

Although  these  results  seem  paradoxical,  they  are  not:  two 
sets  of  figures  need  not  be  identical  (or  even  close)  in  order  to  be 
highly  correlated.  High  positive  correlation  simply  denotes  that 
both  sets  of  figures  slope  in  the  same  direction.  This  suggests  the 
following:  to  the  extent  that  there  are  consistent  upward  biases  in 
the  projected  data,  the  analyses  of  variations  in  costs  are  still 
legitimate.  Unit  cost  estimates,  on  the  other  hand,  should  be 
interpreted  with  caution,  since  they  show  signs  of  upward  bias  on 
the  overall  order  of  ten  percent. 

Another  perspective  on  the  total  sample  cost  characteristics 
is  outlined  in  Table  46.  This  table  compares  physician  and  phy- 
sician assistant  expenditures  to  total  expenditures  for  concurrent 
review  and  MCEs  as  well  as  for  total  review.  Physician  costs  are 
seen  to  be  a  low  percentage  of  concurrent  review  expenditures,  but 
roughly  one-quarter  of  MCE  expenditures.  Physician  assistant 
expenditures,  on  the  other  hand,  are  larger  part  of  concurrent 
review  costs  than  of  MCE  costs. 

There  are  regional  variations  in  these  distributions.  For 
example,  physician  expenditures  as  a  proportion  of  total  review 
expenditures  are  higher  in  the  Northeast  than  in  the  South.  On  the 
other  hand,  Southern  hospitals  spend  a  greater  proportion  of  their 
budgets  on  review  coordination  than  any  of  the  other  regions.  (The 
lowest  region  in  this  regard  is  the  Northeast. ) 

Unit  Cost  Variations  Vs.  Hospital  Characteristics 

The  data  base  developed  for  the  1,185  hospitals  allows  for 
comparison  of  cost  figures  with  hospital  characteristics.  An 
initial  comparison  focused  on  variation  in  unit  costs  in  delegated 
hospitals  according  to  the  following  hospital  characteristics:  (1) 
total  admissions  subject  to  review,  (2)  delegation  status,  (3)  scope 
of  review,  (4)  medical  school  affiliation,  (5)  Standard  Metropolitan 
Statistical  Area  (SMSA)  size,  and  (6)  hospital  ownership.  Sub- 
sequently,  unit  concurrent  review  costs  and  unit  medical  care 
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TABLE  45: 

Comparison  of  Projected  and  Actual  Cost  Data 


No.  of 

Projected 

Actual 

Hospitals 

Correlation 

Average 

Average 

Variable 

Included 

Coefficient 

Cost 

Cost 

Difference 

Admissions 

Medicare 

90 

0.966 

$1,373.78 

$1,351.47 

22.31 

Medicaid 

77 

4. 

0.978 

526.27 

515.71 

11.56 

Non-Federal 

56 

0.986 

3,022.21 

3,015.70 

6.52 

CR  Expenditures 

rnyalClans 

a 

U  «  U  3^7 

I  ,  <tOj  .0  / 

Assistants 

57 

0.855 

2,865.40 

2,852.67 

12.74 

Review 

4. 

Coordinators 

89 

0.926 

9,703.92 

9,160.01 

543.91 

Technical 

4, 

• 

Support 

52 

0.986 

4,549.33 

3,970.12 

579.21 

Fringe  Benefits 

78 

0.941 

2,163.55 

1,984.42 

179.13 

Training 

17 

0.583 

272.12 

610.38 

-338.76 

Travel 

40 

0.527 

177.10 

275.63 

-98.53 

nv^Cj  nxpenuiturss 

Physicians 

40 

0.369+ 

1,022.35 

826.37 

195.98 

Review 

Coordinators 

29 

0.883+ 

1,990.66 

1,657.90 

332.76 

Technical 

Support 

57 

0.8 15+ 

2,816.60 

1,994.58 

822.02* 

Fringe  Benefits 

58 

0.867+ 

547.07 

421.41 

121.66 

Training 

7 

-0.06 

219.29 

155.14 

64.14 

Travel 

15 

0.785+ 

262.20 

143.80 

118.40 

Total  Expenditures 

90 

0.906* 

$20,738.46 

$18,407.42 

$2,331.03* 

_ —  i 

=  correlation  significant  at  5  percent  level 

* 

=  mean  difference  significant  at  5  percent  level 
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TABLE  46: 

Analysis  of  Physician  and  Physician  Assistant 
Costs  for  Delegated  Review  in  Calendar  Year  1977 


Variable  CR  MCE  Total  Review 


Physician  Costs  as  1.7%  26.2%  6.7% 

a  Percent  of  Total 
Costs,  This  Category 

Physician  Assistant 
Costs  as  a  Percent 
of  Total  Costs, 

This  Category  11.4%  1.4%  11.0% 


evaluation  costs  were  compared  with  the  same  characteristics.  The 
relationships  of  all  three  unit  cost  figures  with  the  selected 
characteristics  were  found  to  be  similar.  Consequently,  only  the 
combined  unit  cost  results  are  discussed. 

Unit  costs  per  discharge  were  divided  into  six  distinct 
cost  ranges.  The  cost  ranges  and  percentage  distribution  for  the 
1,185  hospitals  are  shown  below: 


Cost  Range  Percentage  of  Hospitals 

Below  $4  8.4% 

$4  to  $8  29.5 

$8  to  $12  42.5 

$12  to  $16  14.4 
$16  to  $20  3.1 
Over  $20  2.2 


The  first  comparison  of  unit  cost  was  made  against  estimated 
total  admissions  subject  to  review.  The  admission  categories  were 
divided  into  six  categories.  The  six  categories  and  their  associ- 
ated distribution  among  the  1,185  hospitals  are  shown  below: 

Annual  Admissions  Percentage 
Subject  to  Review  Distribution 

Under  1,000  22.5% 

1,000  to  3,000  38.8 

3,000  to  5,000  17.1 

5,000  to  10,000  14.9 

10,000  to  20,000  5.5 

Over  20,000  1.2 
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The  results  show  a  statistically  significant  relation  between  unit 
costs  and  estimated  admissions  subject  to  review.  Generally,  higher 
admission  hospitals  have  lower  unit  costs.  This  suggests  that  there 
exist  some  economies  of  scale  with  respect  to  admissions.  There 
were,  however,  many  hospitals  with  low  unit  costs  and  admissions  of 
under  1,000  or  between  one  and  three  thousand.  This  result  could 
be  caused  by  the  partially  delegated  hospitals  which  would  be 
expected  to  have  lower  unit  total  costs.  The  higher  unit  costs  for 
the  lower  admission  hospitals  could  also  reflect  start-up  costs.  If 
many  of  the  hospitals  in  the  lower  admission  categories  are  just 
beginning  review  activities,  then  as  more  experience  is  gained  and 
admissions  increase,  unit  costs  may  decline.  Hence,  the  cost-admis- 
sion relationship  might  reflect  both  economies  of  scale  and  higher 
costs  for  recently  delegated  hospitals. 

Ninety-two  percent  of  the  hospitals  in  the  data  base  are 
fully  delegated.  These  hospitals  tend  to  have  higher  costs  than 
partially  delegated  institutions.  This  result  should  be  expected 
since  a  partially  delegated  hospital  should  have  fewer  expenditures 
than  a  fully  delegated  hospital. 

Along  with  delegation  status,  the  influence  of  scope  of 
review  on  cost  was  examined.  Scope  of  review  is  identified  as  (1) 
Federal  review  only,  (2)  Federal  and  non-Federal  combined  review 
programs,  and  (3)  Federal  and  non-Federal  separate  programs.  The 
Federal-only  category  contained  nearly  50  percent  of  the  sample  of 
1,185  hospitals.  The  other  two  classifications  divided  the  remain- 
der evenly.  Combined  programs  tend  to  have  lower  unit  costs  than 
Federal-only  or  separate  programs.  This  result  is  statistically 
significant  and  is  consistent  with  the  economies  of  scale  concept 
discussed  early.  Combined  programs  would  have  a  larger  admissions 
base  over  which  costs  could  be  allocated.  However,  an  alternative 
hypothesis  is  that  non-Federal  admissions  might  have  lower  review 
costs  which  would  favorably  affect  the  combined  average.  Additional 
research  would  be  needed  to  verify  the  source  of  the  lower  costs  in 
combined  programs . 

Unit  costs  were  also  compared  for  those  hospitals  with  medical 
school  affiliations  and  nonaffiliated  hospitals.  Medical  school 
affiliated  hospitals  tend  to  have  higher  unit  review  costs.  The 
result  is  statistically  significant  at  the  five  percent  level. 
Medical  school  affiliation  may  act  as  a  case  mix  proxy  or  reflect 
higher  educational  costs  that  flow  into  the  review  process. 

The  influence  of  urban  location  on  unit  cost  was  also  examined. 
Hospitals  were  identified  according  to  Standard  Metropolitan 
Statistical  Area  (SMSA)  size.  The  SMSAs  are  divided  into  six  size 
categories.  Approximately  half  of  the  hospitals  are  located  in 
non-SMSA  areas.  Unit  costs  were  found  to  be  positively  related  to 
SMSA  size  classification.     Highly  urban  areas  have  much  higher  unit 
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costs  than  non-urban  or  smaller  SMSA  areas.  These  results  are 
significant  at  the  five  percent  level  and  possibly  reflect  higher 
wages  and  operating  costs  in  large  metropolitan  areas. 

Ownership  classification  was  also  examined.  The  only  statisti- 
cally significant  relation  found  was  between  unit  cost  and  non- 
Federal  government  hospitals.  These  latter,  municipal  general  hos- 
pitals have  a  proportionally  higher  representation  of  relatively  low 
unit  costs  than  profit  and  other  nonprofit  institutions.  Such  a 
result  may  reflect  either  lower  wage  scales  or  economies  of  scale. 
Further  research  would  more  clearly  identify  the  cause  of  the 
relationship  between  ownership  and  unit  cost. 
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4.     BENEFIT-COST  ANALYSIS  OF  CONCURRENT  REVIEW 


4. 1  Overview 

In  this  section  an  attempt  is  made  to  synthesize  some  of 
the  results  which  bear  on  program  impact  and  program  expendi- 
tures. Through  such  a  synthesis  it  may  be  possible  to  assess 
whether  program  benefits  exceed  their  costs.  While  such  an  analy- 
sis is  appropriate  for  each  program  area  within  PSRO,  available 
data  necessarily  limit  the  focus  of  the  analyses  to  utilization 
control  impacts  achieved  through  concurrent  review./1 


4.1.1     Objectives  and  Study  Relationships 

The  major  objectives  in  this  area  are:  (1)  to  compare  the 
relative  benefits  and  costs  of  utilization  control  through  con- 
current review  and  (2)  to  compare  potential  reductions  in  expen- 
ditures achieved  through  utilization  control  to  the  currently 
observed  rates  of  price  inflation  in  the  health  care  sector.  To 
accomplish  these  goals,  the  study  estimates  the  relation  between 
utilization  changes  and  changes  in  reimbursements  and  tests  the 
sensitivity  of  those  findings  to  various  assumptions.  The  latter 
analysis  is  reported  in  Section  4.2;  the  corresponding  benefit  and 
cost  comparisons  are  presented  in  Section  4.3. 


4.1.2  Data  Sources 

Data  on  real  benefits,  in  terms  of  reduced  days  of  care 
per  1000  eligibles,  derive  from  the  100  Percent  Medicare  Discharge 
File  Study.  Data  sources  for  the  latter  study  were  described 
in  Section  2.  As  the  Medicare  study  is  the  only  available  source  of 
PSRO  utilization  impact  data,  the  program  cost  data  were  adjusted  to 
estimate  expenditures  for  concurrent  review  based  on  Medicare  alone. 
Data  used  to  estimate  the  relation  between  changes  in  Medicare 
utilization  and  changes  in  Medicare  reimbursements  derive  from  a 
file  of  Medicare  cost  reports  which  includes  observations  on  some 
5,200  hospitals  for  1976.  All  data  used  in  the  analysis  were 
adjusted  to  reflect  calendar  year  1977  prices. 

4.1.3  Relationship  to  the  1977  Evaluation 

This  benefit-cost  analysis  of  concurrent  review  follows 
directly  from  a  corresponding  analysis   in  the   1977  evaluation. 


/1  An  examination  of  potential  benefit-cost  ratios  of  medical 
care  evaluation  studies  has  been  initiated,  with  results 
expected  in  1979  (see  Section  5). 
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The  differences  between  that  study  and  the  current  one  are  as 
follows:  (1)  a  greater  amount  of  data  was  available  for  the  pre- 
sent study,  increasing  the  number  of  PSROs  in  the  study  more  than 
tenfold;  (2)  the  estimating  algorithm  was  refined  to  allow  more 
detailed  estimates  than  last  year;  (3)  the  results  were  tested 
in  terms  of  their  sensitivity  to  assumptions  about  substitution 
effects  outside  the  hospital  sector;  and  (4)  assumptions  about  the 
kinds  of  hospitals  which  favor  utilization  reductions  were  entered 
into  the  analysis  to  assess  their  effects  on  the  resulting  esti- 
mates. 

The  cost  estimates  of  this  analysis  differ  from  last  year's 
in  two  respects.  First,  since  traditional  utilization  review 
can  no  longer  be  considered  an  alternative  to  PSRO,  the  full  cost 
estimates  of  the  program  are  used  rather  than  the  incremental 
cost  estimates  with  respect  to  utilization  review.  Second,  since 
startup  costs  can  be  considered  as  fixed,  the  current  comparisons 
will,  to  the  extent  possible,  be  based  primarily  on  operating 
costs. 


4.2    Expenditure  Impact  Analysis 

4.2.1  Objectives 

The  objectives  of  this  analysis  are:  (1)  to  estimate  the 
relation  between  changes  in  Medicare  hospital  utilization  and 
changes  in  Medicare  reimbursements  for  health  care  services;  and 
(2)  to  test  the  sensitivity  of  these  estimated  relations  to  changes 
in  the  assumptions  used  in  the  analysis. 

4.2.2  Data 

The  primary  source  of  data  for  this  analysis  is  a  file  of 
audited  Medicare  cost  reports  for  fiscal  year  1976.  This  file 
contains  information,  by  provider,  on  days  of  care  and  expendi- 
tures for  routine  and  ancillary  services  for  both  the  total  patient 
population  and  for  Medicare  patients  alone. 

These  data  were  adjusted  to  reflect  calendar  year  1977  by 
using  a  uniform  inflation  factor  for  the  fifteen-month  differ- 
ence between  the  cost  data  and  the  utilization  data.  The  figure 
used  was  the  change  in  the  per  diem  Medicare  reimbursement  from 
"old"  fiscal  year  1976  to  "new"  fiscal  year  1977.  The  Medicare 
Actuary's   Office  has    indicated  that  this   figure   is    17.9  percent. 

State  data  on  skilled  nursing  facility  per  diem  charges  for 
Medicare  are  also  incorporated  into  the  analysis  to  allow  esti- 
mates of  substitution  effects   involving  the  long-term  care  sector. 
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4.2.3  Methods 


The  methodology  applied  in  deriving  the  estimates  for  this 
analysis  is  a  form  of  simulation.  At  the  hospital  level,  the 
question  is  asked:  How  would  Medicare  reimbursements  for  routine 
and  ancillary  services  change  given  a  prescribed  change  in  Medicare 
days  of  care?  Various  weighting  schemes  are  tested  to  assess  the 
sensitivity  of  these  estimated  changes  to  assumptions  about  the 
types  of  hospitals  where  utilization  changes  are  more  likely.  In 
addition,  given  the  partitioning  of  the  various  subcomponent  im- 
pacts, a  variety  of  scenarios  about  possible  gross  changes  in 
reimbursements  was  examined.  The  estimating  algorithm  is  de- 
scribed below. 

The  basis  of  the  algorithm  is  a  partitioning  of  the  changes 
in  Medicare  health  service  reimbursements  that  would  actually  take 
place  were  there  to  be  changes  in  Medicare  days  of  care  in  short- 
term  hospitals.  At  an  initial  level  there  are  six  distinct  sub- 
components associated  with  such  utilization  changes.  Four  of  these 
are  associated  with  changes  in  reimbursements  to  the  hospital;  the 
other  two  involve  changes  in  reimbursements  to  non-hospital  pro- 
viders. 

The  subcomponents  which  involve  non-hospital  providers  are 
substitution  effects.  They  involve  substitution  of  health  services 
outside  the  hospital  for  services  that  might  otherwise  have  been 
rendered  on  an  inpatient  basis.  For  example,  certain  diagnostic 
tests  may  be  performed  on  an  ambulatory  basis  rather  than  in  the 
hospital.  Alternately,  a  patient  whose  hospital  stay  is  shortened 
may  spend  a  greater  amount  of  time  in  a  long-term  care  facility.  In 
either  event,  there  is  an  additional  cost  element  involved  which 
offsets,  at  least  partially,  the  days  of  acute  care  saved.  The 
algorithm  assumes  both  an  ambulatory  care  and  a  long-term  care 
substitution  effect. 

As  stated,  four  separate  changes  in  hospital  reimbursements 
were  considered.  They  can  be  characterized  along  two  dimensions. 
The  first  dimension  simply  follows  the  Medicare  distinction  between 
routine  services  and  ancillary  services.  The  second  dimension  is 
somewhat  more  complicated.  Any  change  in  Medicare  reimbursements 
due  to  a  change  in  utilization  consists  of  two  separate  impacts.  The 
first  is  a  reallocation  of  existing  hospital  expenditures  because  of 
a  change  in  the  proportion  of  total  days  of  care  that  are  Medicare 
days.  The  second  is  a  change  in  the  hospital's  decisions  about 
hiring,  purchases  of  supplies  and/or  other  factor  inputs  brought 
about  as  a  result  of  changes  in  utilization.  If  such  input  purchase 
changes  occur,  they  would  lead  to  a  change  in  Medicare  reimburse- 
ments equal  to  the  apportioned  change  in  input  expenditures.  Hence 
for  both  ancillary  and  routine  services  there  will  be  both  a  reallo- 
cation effect  and  an  input  purchase  effect. 
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The  "simulation"  begins  at  the  level  of  the  individual  hos- 
pital. It  asks  the  following  question:  Assuming  a  certain  per- 
centage change  in  days  of  care  provided  by  a  given  hospital,  what 
are  the  resulting  changes  in  the  four  hospital  reimbursement  sub- 
components? The  relation  between  reimbursement  changes  and  util- 
ization changes  is  expectedly  nonlinear;  therefore  several  discrete 
estimates  were  prepared  for  selected  potential  utilization  decreases 
(or  increases)  of  up  to  twenty-one  percent.  Fourteen  such  values 
were  postulated  in  all. 

Since  it  is  not  possible  to  attribute  changes  in  days  of 
care  to  a  specific  hospital  in  a  PSRO  area,  the  individual  hospital 
effects  are  aggregated  to  the  PSRO  level.  The  first  method  of 
aggregation  assumed  that  changes  in  Medicare  days  of  care  were 
distributed  across  hospitals  as  the  days  themselves  were  distri- 
buted. In  other  words,  all  hospitals  were  assumed  to  experience  an 
equal  percentage  change  in  days  of  care.  This  was  the  approach  used 
in  last  year's  report. 

There  are  a  number  of  plausible  alternative  hypotheses,  how- 
ever, about  how  PSRO  utilization  changes  might  be  distributed. 
These  were  simulated  by  weighting  the  days  of  care  in  each  hospital 
by  additional  hospital  characteristics.  Five  such  weighting  schemes 
were  employed  in  the  simulation  to  assess  the  sensitivity  of  the 
estimates  to  assumptions  about  which  hospitals  were  more  likely  to 
be  influenced  by  PSRO  review.  These  alternative  weights  included 
Medicare  average  length  of  stay,  cost  per  discharge,  and  hospital 
occupancy  rates  relative  to  the  average  for  the  PSRO  area. 

The  simulation  to  this  point,  then,  involves  four  subcomponent 
estimates,  for  each  of  fourteen  discrete  potential  utilization 
changes  between  plus  and  minus  twenty-one  percent,  for  each  of  six 
alternative  weighting  schemes,  for  each  PSRO  area. 

The  remaining  complication  involved  combining  the  subcomponent 
estimates  with  the  substitution  effects.  In  the  1977  PSRO  Evalua- 
tion Report,  two  point  estimates  of  combined  effects  were  computed. 
These  two  estimates  were  also  computed  for  this  year's  analysis.  In 
addition,  ten  other  sets  of  estimates  were  computed  involving 
various  assumptions  about  the  likely  combinations  of  subcomponents 
and  the  magnitude  of  the  substitution  effects. 

Routine  services  reallocation  was  assumed  to  occur  under 
all  circumstances.  The  other  alternatives  considered  involved 
either  (i)  routine  input  purchase  effects,  (ii)  ancillary  reallo- 
cation effects,  (iii)  ancillary  input  purchase  effects,  or  (iv)  some 
combination  of  the  above.  For  example,  following  one  of  last 
year's  hypotheses,  total  ancillary  service  usage  per  episode  might 
remain  constant  despite  a  reduction  in  average  length  of  stay. 
Alternately,  reallocation  effects  might  be  observed  for  ancillary 
services  (as  well  as  for  routine),  but  there  might  not  be  input 
purchase  effects  for  either  form  of  service. 
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4 . 2 . U  Findings 


In  the  96  PSRO  areas  studied,  the  Medicare  per  diem  short 
stay  hospital  reimbursement  in  CY  1977  was  estimated  to  be  $169.75. 
This  ranged  from  a  low  of  $93.62  in  Mississippi  to  a  high  of  $260.68 
in  Alaska.  (San  Francisco  was  the  second  highest  at  $250.48.) 
The  average  reimbursement  per  discharge  was  $1,758.82,  with  lows 
and  highs,  respectively,  of  $882.39  in  Arkansas  and  $3,630.94 
in  Manhattan. 

These  estimates  were  generated  as  a  first  product  of  the 
simulation  used  to  assess  the  relation  between  changes  in  short 
stay  hospital  utilization  and  changes  in  Medicare  reimbursements. 
As  indicated,  the  simulation  was  based  on  six  separate  subcomponent 
changes  in  reimbursements:  two  substitution  effects,  two  realloca- 
tion effects,  and  two  input  purchase  effects.  An  examination  of 
each  of  these  effects,  in  turn,  demonstrates  the  sensitivity  of  the 
simulation  to  changes  in  assumptions.  These  are  illustrated  in 
terms  of  changes  induced  in  the  estimates  of  the  value  of  a  day 
saved  as  a  percentage  of  the  average  per  diem  hospital  reimburse- 
ment. 

Substitution  Effects  -  Estimates  of  ambulatory  and  long-term 
care  substitution  effects  were  based  on  per  diem  ancillary  service 
reimbursements  to  short  stay  hospitals,  and  long-term  care  reim- 
bursements. These  averaged  $62.37  and  $62.41,  respectively.  Table 
47  illustrates  the  range  of  these  estimates  in  both  absolute  dollar 
amounts  and  as  percentages  of  total  short  stay  hospital  per  diem 
reimbursements . 

TABLE  47 

RANGES  OF  MEDICARE  PER  DIEM  ANCILLARY  AND  LONG-TERM  CARE  REIMBURSEMENTS 


Form  in  Which  Reim- 

Corresponding 

Subcomponent 

bursement  is  Stated 

Amount 

PSRO 

Anci llary 

Per  Diem 

High 

$109.14 

OAL01 

(in  dollars) 

Low 

30.44 

2NY05 

Average 

62.37 

Per  Diem  as  %  of 

High 

46.8% 

9CA04 

Total  Short  Stay 

Low 

28.6% 

2NY16* 

Hospital  Per  Diem 

Average 

36.7% 

Long-Term 

Per  Diem 

High 

$  81.35 

OAL01 

Care 

(in  dollars) 

Low 

33.01 

OID01 

Average 

62.41 

Per  Diem  as  %  of 

High 

79.3% 

4MS01 

Total  Short  Stay 

Low 

23.1% 

3DC01 

Hospital  Per  Diem 

Average 

36.8% 

*Cost  data  from  2NY16  are  based  on  approximately  40%  of  the  Medicare 
days  of  care  and  may  be  subject  to  error. 
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A  long-term  care  substitution  effect  was  assumed  to  occur  under 
all  the  alternatives.  Since  no  empirical  estimates  of  this  effect 
were  available,  a  variety  of  estimates  were  employed  ranging  from  0 
to  .25.  (These  estimates  were  derived  from  discussions  with  HCFA 
personnel  involved  in  long-term  care.  Only  about  seven  percent  of 
Medicare  short-term  hospital  stays  are  followed  by  long-term  care 
stays  so  the  assumed  range  of  long-term  care  substitutions  is 
reasonably  conservative.) 

Empirical  estimates  of  ambulatory  care  substitution  effects 
were  also  not  available.  The  simulation  in  this  case  involved 
plausible  estimates  of  that  effect.  Following  discussions  with  HCFA 
personnel,  several  expenditure  substitution  ratios  were  postulated, 
ranging  from  .25  to  1.0.  Available  Medicare  statistics  suggest  that 
ancillary  service  utilization  per  day  begins  to  level  off  after  the 
initial  week  of  a  hospital  stay.  At  that  point,  the  estimated  daily 
utilization  of  ancillary  services  does  not  differ  appreciably  from 
the  per  diem  usage  for  an  entire  episode.  For  this  reason,  the 
estimated  per  diem  ancillary  service  reimbursement  was  multiplied  by 
the  assumed  ambulatory  care  substitution  ratios  to  estimate  the 
ambulatory  care  substitution  effect. 

The  results  of  the  various  simulations  consist  of  a  set  of 
"best  case"  and  "worst  case"  estimates  with  respect  to  potential 
changes  in  reimbursements  due  to  changes  in  utilization. 

There  exist  no  data  on  utilization  changes  of  non-Medicare 
patients  due  to  PSRO  involvement.  Therefore,  for  the  purpose 
of  developing  an  estimating  algorithm,  it  was  assumed  that  there 
was  no  change.  This  introduces  a  potential  bias  in  the  analysis. 
Unfortunately,  the  direction  of  the  bias  is  not  unambiguous.  If 
non-Medicare  (as  well  as  Medicare)  utilization  falls,  the  Medicare 
savings  estimate  is  biased  upward.  On  the  other  hand,  if  hospitals 
respond  to  decreases  in  Medicare  utilization  by  increasing  the 
utilization  of  other  patient  populations,  then  the  savings  estimates 
would  be  biased  downward. 

Finally,  it  should  be  noted  that  there  is  a  potential  missing 
data  problem  in  that  not  all  hospitals  which  participate  in  Medicare 
are  included  in  the  machine-readable  file  of  cost  reports.  There 
was  a  concern  last  year  that  most  of  the  hospitals  without  data  were 
relatively  large,  thus  biasing  the  cost  estimates  downward.  Given 
the  large  number  of  hospitals  involved  in  this  year's  analysis,  it 
was  not  possible  to  analyze  the  omitted  hospitals  in  time  for  this 
report.  Nonetheless,  the  overall  bias  from  these  omissions  should 
be  small.  In  only  eight  of  the  ninety-six  PSRO  areas  in  the  dis- 
aggregated analysis  did  missing  hospitals  represent  more  than  fifty 
percent  of  the  Medicare  days  of  care.  Among  those  eight  areas,  two 
had  cost  reports  from  more  than  fifty  percent  of  their  hospitals. 
The  six  remaining  areas  with  potentially  serious  data  problems  were 
1CT01,   1CT02,   1CT04,   1RI01 ,  2NY16,  and  8UT01. 
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Because  only  about  seven  percent  of  Medicare  hospitaliza- 
tions are  followed  by  long-term  stays,  the  long-term  care  sub- 
stitution ratio  was  assigned  generally  low  values  —  between  zero 
and  0.25.  The  value  used  in  the  final  simulation  was  0.15;  that  is, 
for  every  100  days  of  short  stay  care  averted,  there  would  be  an 
assumed  additional  15  days  of  long-term  care. 

Selection  of  values  for  the  ambulatory  care  substitution 
ratio  was  more  problematic.  One  might  argue  that  in  the  kinds 
of  cases  where  PSROs  could  appropriately  reduce  utilization,  cur- 
rently observed  ancillary  usage  could  be  reduced  with  little  or 
no  substitution  of  ambulatory  care.  This  suggests  a  low  ambula- 
tory care  substitution  ratio.  On  the  other  hand,  one  could  argue 
that  economies  of  scale  make  hospitals  relatively  efficient  pro- 
viders of  the  services  in  question.  If  the  prices  of  services 
in  the  hospital  are  significantly  lower  than  the  same  services 
provided  on  an  ambulatory  basis,  then  a  high  value  of  the  substitu- 
tion ratio  should  be  used.  A  second  argument  can  be  made  for 
using  a  relatively  high  substitution  ratio  in  the  simulation.  This 
argument  is  based  on  the  hypothesis  that  total  ancillary  usage  per 
episode  may  not  change  appreciably  as  a  result  of  PSRO  review;  that 
is  to  say,  existing  ancillary  usage  may  merely  be  compressed  into 
shorter  lengths  of  stay  but  not  fundamentally  reduced. 

This  uncertainty  would  not  be  much  of  a  problem  if  ancillary 
utilization  were  not  a  significant  part  of  total  hospital  usage. 
The  latter  is  not  the  case,  however;  estimates  of  the  value  of  a  day 
saved  are  very  sensitive  to  changes  in  the  assumed  substitution 
ratio.  If  this  ratio  is  assumed  to  be  zero,  the  value  of  a  day 
saved  is  fifty-four  percent  of  the  average  per  diem.  On  the  other 
hand,  if  a  one-to-one  substitution  ratio  in  ancillary  care  expendi- 
tures is  assumed,  then  the  value  of  a  day  saved  falls  to  less  than 
eighteen  percent  of  the  average  per  diem.  A  middle  ground  sub- 
stitution ratio  estimate  of  0.5  was  assumed  for  the  final  simula- 
tion, but  clearly  this  value  cannot  be  considered  definitive  in  the 
absence  of  other  research.  (The  assumed  ratio  of  0.5  implies  that 
for  every  dollar  reduction  in  Medicare  ancillary  services  reim- 
bursement, there  will  be  a  fifty  cent  increase  in  reimbursements  for 
ambulatory  care.) 

Reallocation  Effects  -  Of  the  remaining  sets  of  subcomponents, 
the  reallocation  effects  are  the  more  significant.  The  average 
routine  service  reallocation  effect  was  $63.09  per  day  saved  and  the 
average  ancillary  reallocation  efect  was  $38.14.  Table  48  presents 
the  ranges  in  these  estimates  across  PSROs.  If  there  were  no 
ancillary  reallocation  effects  (and  no  ambulatory  substitution), 
then  the  value  of  a  day  saved  would  be  about  thirty-two  percent 
of  the  average  per  diem;  this  figure  comes  about  by  subtracting  from 
the  37.1%  savings  due  to  routine  services  reallocation  the  5.5%  of 
additional  cost  due  to  long-term  care  substitution  (i.e.,  36.8%  x 
0.15).  Adding  both  ancillary  reallocation  and  ambulatory  substitu- 
tion effects  (at  plus  22.5%  and  minus  18.4%  respectively)  increases 
the  savings  estimate  to  roughly  36  percent. 
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TABLE  48 

RANGES  OF  REALLOCATION  EFFECTS  PER  DAY  SAVED 


Subcomponent 

Form  in  Which 
Savings  is  Stated 

Amount 

Corresponding 
PSRO 

Routine 
Services/ 1 

Per  Diem 
(in  dollars) 

High  $121.33 
Low  29.75 
Average  63.09 

OAK01 
7M04 

Per  Diem  as  %  of 
Total  Short  Stay 
Hospital  Per  Diem 

High  47.8% 
Low  26.4% 
Average  37.1% 

50H12 
4FL12 

Ancillary 
Services 

Per  Diem 
(in  dollars) 

High          $  94.97 
Low  16.53 
Average  38.14 

OAK01 
2NY05 

Per  Diem  as  %  of 
Total  Short  Stay 
Hospital  Per  Diem 

High  36.4% 
Low  15.9% 
Average  22.5% 

OAK01 
1VT01 

Input  Purchase  Effects  -  Input  purchase  effects  would  occur 
if  changes  in  Medicare  utilization  led  to  changes  in  hospitals' 
expenditures  for  equipment,  supplies,  personnel,  etc.  Although  the 
inclusion  of  such  effects  would  increase  the  estimates  of  the  value 
of  a  day  saved,  these  increases  would  not  change  the  estimates 
appreciably.  To  begin  with,  the  conventional  approximation  of  these 
variable  costs  is  only  forty  percent  of  the  average  per  diem. 
Further,  even  if  there  were  changes  in  these  expenditures,  Medicare 
would  only  receive  its  apportioned  share  —  about  one-third  —  of 
the  reduction.  The  values  of  potential  input  purchase  effects  for 
routine  and  ancillary  services  were  $16.49  and  $9.68,  respectively 
—  minor  compared  to  the  substitution  and  reallocation  effects. 
Given  the  relatively  small  change  in  utilization  observed  in  the 
96  study  areas  (by  which  these  effects  must  be  multiplied) ,  the 
potential  input  purchase  effects  were  not  included  in  the  final 
simulation. 

By  construction,  both  of  the  substitution  effects  are  linear; 
that  is,  for  every  day  saved  there  is  a  constant  substitution 
of  ambulatory  care  and  long-term  care.  The  reallocation  effects, 
however,  are  nonlinear.  As  Medicare  utilization  changes,  the 
value  of  a  day  saved  changes  as  well.  A  specific  example  is 
presented  in  Table  49.  The  Foundation  for  Medical  Care  Evaluation 
of  Southeastern  Wisconsin  —  5WI02  ■ —  was  selected  because  its 
average  per  diem  Medicare  reimbursement  and  its  estimated  impact 
were  closest  to  the  average  of  the  96  areas. 


/1      Reallocation  of  routine  services,  as  noted  earlier,  is  assumed 
to  take  place  under  all  circumstances. 
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TABLE  49 

RELATIONSHIP  BETWEEN  UTILIZATION  CHANGES 
AND  VALUE  OF  A  DAY  SAVED 


Value  of  Day  Saved 

Utilization  Change 

6 
V 

v  or  Average  irej.  Diem 

%  Reduction  in 

21% 

$72.69 

44.29% 

Medicare  Days 

13 

CO     O  O 
D3  (JO 

ylO  *5"7 

of  Care 

8 

£■7  An 

D  /  •  f»U 

ji  1  fi"7 
4  I  •  U  / 

5 

DO  •  ZD 

•  J  / 

3 

DO  •  DU 

•  y  i 

2 

DStlO 

1 

Oft 

%  Increase  in 

1% 

$64.03 

39.02% 

Medicare  Days 

2 

63.07 

38.80 

of  Care 

3 

63.31 

38.58 

5 

62.60 

38.58 

8 

61.55 

37.51 

13 

59.85 

36.47 

21 

57.25 

34.89 

NOTE:  This  example  is  based  on  the  experience  of  PSRO  5WI02. 


This  pattern  was  repeated  in  all  of  the  PSRO  areas.  The 
greater  the  reduction  in  Medicare  days  of  care,  the  greater  the 
value  of  each  day  saved,  with  a  reverse  trend  holding  in  the  case 
of  utilization  increases.  These  results  from  the  simulation  allowed 
the  estimate  of  the  value  of  a  day  saved  to  be  adjusted  according  to 
the  estimated  impact  on  days  of  care  for  each  individual  PSRO 
area. 

The  values  presented  thus  far  were  based  on  a  simulation 
run  that  assumed  that  reductions  in  days  of  care  would  be  distri- 
buted across  hospitals  as  the  days  of  care  were  distributed.  That 
is,  if  one  hospital  had  half  of  a  PSRO  area's  Medicare  days  of 
care,  it  would  also  have  half  of  the  days  saved  in  that  area.  Since 
reductions  in  days  of  care  cannot  be  attributed  to  a  specific 
hospital,  the  equal  distribution  assumption  is  a  fairly  neutral 
one.  One  might  argue,  however,  that  there  are  certain  character- 
istics of  hospitals  which  would  make  utilization  reductions  more  or 
less  likely  for  some  hospitals  than  for  others.  Simulations  were 
therefore  conducted  using  alternative  assumptions  to  assess  the 
sensitivity  of  the  savings  estimates  to  these  changes. 

Each  alternative  involved  weighting  the  days  of  care  by  an 
index  based  on   each   hospital's   average    length   of   stay,    cost  per 
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discharge,  or  occupancy  rate  with  respect  to  all  of  the  hospitals  in 
the  PSRO  area.  In  effect,  each  alternative  involved  the  following 
type  of  question:  What  would  have  been  the  savings  in  a  PSRO  area 
if  days  of  care  reductions  had  been  relatively  more  concentrated  in 
hospitals  with,  for  example,  a  high  Medicare  average  length  of 
stay? 

The  five  nonneutral  alternatives  included  high  ALOS,  high 
cost  per  discharge,  low  cost  per  discharge,  high  occupancy  rate, 
and  low  occupancy  rate.  The  cost  per  discharge  alternatives  were 
included  to  bracket  the  savings  estimates.  Clearly  PSROs  will 
not  be  able  to  save  days  only  in  high  cost  hospitals  or  only  in 
low  cost  hospitals.  These  estimates,  however,  should  give  an 
upper  and  lower  bound  to  the  savings  estimates. 

The  occupancy  rate  alternatives  were  included  to  assess  the 
effects  of  competing  hypotheses  about  effective  utilization  con- 
trol programs.  One  hypothesis  argues  that  high  occupancy  rate 
hospitals  had  reduced  unnecessary  utilization  prior  to  the  advent 
of  PSROs  so  reductions  will  be  more  likely  in  low  occupancy  rate 
hospitals.  Conversely,  it  is  argued  that  low  occupancy  rate 
hospitals  will  resist  efforts  to  reduce  utilization,  hence  savings 
will  be  more  likely  in  the  high  occupancy  rate  hospitals. 

Based  on  the  neutral  weighting  assumption,  the  average  value  to 
Medicare  of  a  day  saved  would  have  been  $60.67  or  36  percent  of  the 
average  per  diem.  Using  the  1977  approach  (which  did  not  include 
substitution  effects)  would  have  produced  an  estimate  of  $62.11  or 
37  percent  of  per  diem. 

These  results  were  found  to  be  moderately  sensitive  to  changes 
in  the  weighting  assumptions,  but  there  was  no  assumption  which 
produced  a  consistent  change  across  all  PSROs.  For  example,  con- 
centrating utilization  reductions  in  high  cost  per  discharge  hos- 
pitals raises  the  value  of  a  day  saved  by  about  ten  percent  to 
$67.11.  A  concentration  of  reduction  in  low  cost  hospitals  would 
reduce  the  value  of  a  day  saved  by  nine  percent  to  $54.86.  How- 
ever, there  were  seven  PSROs  that  would  have  had  an  increase  in  the 
value  of  a  day  saved  if  utilization  reductions  had  been  more  con- 
centrated in  relatively  low  cost  hospitals. 

Concentrating  on  hospitals  with  relatively  high  average 
lengths  of  stay  increases  the  value  of  a  day  saved  by  one  per- 
cent to  $61.38.  However,  in  thirty-four  of  the  PSROs,  the  value 
of  a  day  saved  would  have  been  reduced.  Concentrations  with  re- 
spect to  occupancy  rates  sometimes  had  paradoxical  results.  In 
some  areas,  the  value  of  a  day  saved  was  increased  regardless  of 
whether  the  high  or  low  occupancy  alternative  was  examined.  And  in 
Riverside,  California  --  9CA27  —  both  options  led  to  reductions. 
In  the  aggregate,  however,  changes  based  on  the  occupancy  rate 
assumption  were  small.  The  value  of  a  day  saved  was  $61.04  for  high 
occupancy;   $60.67  for  low  occupancy. 
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Because  the  length  of  stay  and  occupancy  rate  alternatives 
did  not  markedly  change  the  estimates,  the  final  simulation  employed 
the  neutral  weighting  by  relative  days  of  care. 

4.3    Benefit-Cost  Comparison 

4.3.1  Objectives 

The  objectives  of  a  complete  benefit-cost  study  would  involve 
an  examination  of  the  ratios  of  marginal  benefits  to  marginal 
costs  to  assess  individual  programs*  relative  efficiencies.  As 
previously  noted,  however,  only  a  restricted  benefit-cost  study  is 
possible  due  to  lack  of  complete  data  on  the  benefit  side.  In 
addition,  given  the  expected  change  to  focused  review  in  the  future, 
it  is  not  clear  that  there  is  an  appropriate  framework  from  which  to 
judge  marginal  costs  of  the  program. 

For  these  reasons,  relatively  simple  benefit-cost  comparisons 
were  the  primary  objectives  of  this  portion  of  the  study.  The 
estimated  utilization  changes  due  to  PSRO  review  were  coupled  with 
the  expenditure  impact  study  results  (Section  4.2)  to  estimate  the 
total  benefits  for  the  nation  and  for  each  active  PSRO  area.  These 
figures  were  compared  to  the  following  estimates  of  costs:  (1)  the 
100  percent  review  figures  calculated  for  1977;  and  (2)  the  costs 
that  would  have  been  incurred  if  concurrent  review  could  have  been 
conducted  at  a  unit  cost  of  $8.70. 

One  final  objective  was   to  put  these  results   in  the  context 

of   currently   observed  increases  in  health   care   prices.      In  this 

way  it  may  be  possible  to  assess  the  potential  of  concurrent  review 
for  cost  containment. 


4.3.2  Data  Sources 

Data  for  this  analysis  came  from  several  sources.  Estimated 
changes  in  utilization  derive  from  the  analyses  of  the  100  percent 
Medicare  discharge  file,  described  in  Section  2.  Estimates  of 
program  costs  come  from  studies  described  in  Section  3.  Linking 
the    two    are    the    expenditure    impact    simulations    just  described. 

4.3.3  Methods 

Given  the  changing  status  of  concurrent  review,  only  rela- 
tively simple  benefit-cost  comparisons  are  presented  in  this  sec- 
tion of  the  report.  Since  concurrent  review  would  "pay  for  itself" 
if  benefits  and  costs  were  equal,  a  variety  of  situations  will  be 
explored  to  determine  "breakeven"  costs  and  benefits  given  certain 
assumptions  about  program  status. 
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The  costs  for  concurrent  review  for  Medicare  only  were  synthe- 
sized using  the  previous  analyses  of  delegated  hospital  review  costs 
and  the  PMIS  data  on  unit  costs  of  review.  The  direct  costs  of 
concurrent  review  were  estimated  by  using  a  weighted  average  of  the 
unit  costs  of  delegated  concurrent  review  and  the  unit  costs  of 
nonde legated  review. 

Management  and  support  costs  were  allocated  to  concurrent 
review  using  a  two  step  process:  First,  for  those  PSROs  conducting 
demonstrations  in  long-term  care  or  ambulatory  care  review,  manage- 
ment and  support  costs  were  reduced  by  a  factor  equal  to  the  per- 
centage of  the  review  costs  that  were  attributed  to  the  demonstra- 
tions. Thus,  if  a  PSRO  had  spent  five  percent  of  its  total  expendi- 
tures for  review  on  a  long-term  care  demonstration,  its  management 
and  support  costs  would  be  reduced  by  five  percent.  In  the  second 
step,  management  and  support  costs  were  multiplied  by  the  ratio 
of  concurrent  review  expenditures  to  the  sum  of  concurrent  review 
expenditures  and  MCE  expenditures.  Thus,  if  five  dollars  were  spent 
on  concurrent  review  for  every  dollar  spent  on  MCEs,  five-sixths 
of  management  and  support  costs  would  be  allocated  to  concurrent 
review.  (Areawide  review  costs  were  also  allocated  to  concurrent 
review  using  this  latter  step.) 

In  order  to  adjust  the  FY  1977  estimates  to  calendar  year 
estimates,  two  corrections  were  applied.  A  1.3  percent  inflation 
adjustment  was  made  to  correct  for  increases  in  prices  as  indicated 
by  the  Consumer  Price  Index.  A  second  correction  was  applied 
to  account  for  the  increase  in  discharges  subject  to  review  between 
FY  1977  and  CY  1977.  This  adjustment  was  accomplished  using  either 
the  estimated  percentage  increase  in  Medicare  discharges  subject 
to  review  or  the  percentage  increase  in  total  program  expenditures 
reported  in  the  PMIS.  (In  the  PSROs  which  had  substantial  increases 
in  the  number  of  discharges  subject  to  review,  these  adjustments  may 
have  led  to  estimates  which  were  biased  upward.  The  bias  imparted 
to  the  aggregate  total,  however,  should  be  small.) 

Finally,  these  numbers  were  multiplied  by  the  percentage 
of  Federally-reviewed  patients  who  were  Medicare  beneficiaries 
to  get  an  estimate  of  the  costs  of  concurrent  review  for  Medicare 
only.     For  the  96  areas,  this  total  came  to  $45,831,466. 


4.3.4  Limitations 

Since  this  analysis  involves  synthesis  of  many  of  the  earlier 
studies,  its  results  are  somewhat  attenuated.  The  major  limitation, 
however,  derives  from  the  lack  of  data  on  the  costs  and  benefits  of 
focused  review.  Since  concurrent  review  is  undergoing  a  significant 
change  in  methodology,  the  results  of  the  current  analysis  can  only 
be  a  historical  snapshot  of  the  relative  efficiency  of  concurrent 
review  in  1977.     Certain  inferences  can  be  drawn  about  the  level  of 
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efficiency  needed  in  focused  review,  but  more  definitive  statements 
must  await  another  year  of  data. 


4 « 3 . 5  Findings 

The  utilization  studies  found  an  aggregate  reduction  in  days 
of  care  of  1.5  percent.  If  this  percentage  were  applied  equally  to 
all  PSROs  in  the  study,  the  resulting  savings  to  Medicare  would  have 
been  $40,857,651.  The  aggregate  benefit-cost  ratio  in  that  event 
would  have  been  0*892,  although  thirty-one  PSROs  would  have  had 
benefit-cost  ratios  greater  than  one.  Roughly,  a  uniform  1.7 
percent  reduction  across  all  PSROs  would  have  been  needed  in  order 
to  breakeven.  (In  the  1977  Evaluation,  it  was  estimated  that  a 
uniform  2.05  percent  reduction  would  have  been  required  to  break- 
even. Medical  care  prices  have  been  increasing  faster  than  the  cost 
of  concurrent  review. ) 

The  results  from  the  disaggregated  analysis,  however,  show 
that  all  PSROs  did  not  have  the  same  impact.  Using  individual 
PSRO  impact  produces  an  estimated  savings  of  $50,473,302,  raising 
the  nationwide  benefit-cost  ratio  to  1.10.  The  number  of  PSROs  with 
benefit-cost  ratios  greater  than  1.0  remained  essentially  unchanged 
at  thirty.  Brooklyn  and  Manhattan  in  New  York  City  had  the  highest 
benefit-cost  ratios,   11.9  and  10.7  respectively. 

The  difference  in  results  between  using  the  uniform  impact 
estimates  and  the  disaggregated  impact  estimates  lies  in  the 
differences  among  PSRO  areas  in  the  estimated  value  of  days 
saved.  On  the  whole,  areas  with  the  greatest  impact  had  relatively 
expensive  days  and  those  with  negative  impact  (increased  days  of 
care)  had  relatively  inexpensive  days.  Although  this  runs  counter 
to  last  year's  results,  there  was  no  significant  correlation  between 
impact  and  per  diem  costs  in  either  direction.  There  were  areas  with 
relatively  expensive  days  which  experienced  little  or  no  impact  and 
areas  with  relatively  inexpensive  days  which  had  considerable 
impact. 

The  two  approaches  permit  an  estimate  of  what  the  aggre- 
gate impact  would  have  been  if  all  below-average  PSROs  could  have 
been  brought  up  to  at  least  the  average  impact  of  1.5  percent. 
Were  this  to  be  accomplished,  52  of  the  96  areas  would  have  had 
benefit-cost  ratios  in  excess  of  one  and  the  aggregate  benefit-cost 
ratio  would  have  been  1.51. 

For  fiscal  year  1979,  the  target  unit  cost  for  total  hospital 
review  is  $8.70  compared  to  this  year's  average  of  $12.80.  To 
achieve  this  target  will  require  a  reduction  in  the  unit  cost  of 
concurrent  review  of  about  $2.25.  If  this  could  be  accomplished  in 
the  96  areas  without  loss  of  impact,  the  resulting  benefit-cost 
ratio  would  be  1.35. 
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One  final  set  of  estimates  was  computed  to  assess  the  poten- 
tial for  concurrent  review  as  a  cost  control  measure.  As  noted 
earlier,  the  average  reimbursement  per  discharge  in  the  96  areas 
studied  was  $1,758.82.  The  cost  of  concurrent  review  was  roughly 
three-quarters  of  one  percent  of  the  cost  of  hospital  care.  Even  if 
the  PSRO  program  had  a  two  to  one  aggregate  benefit-cost  ratio  and 
there  were  no  other  forces  acting  to  increase  hospital  prices,  PSRO 
alone  could  effect  only  a  1.5%  reduction  in  Medicare  expenditures. 
In  fact,  the  benefit-cost  ratio  observed  was  1.1.  Under  extremely 
favorable  circumstances,  this  would  have  led  to  a  reduction  in 
Medicare  expenditures  of  only  .8  percent. 

The  circumstances,  however,  are  not  extremely  favorable. 
There  are  a  host  of  other  factors  which  will  tend  to  increase 
Medicare  expenditures  regardless  of  the  PSRO  program.  In  FY  1978, 
the  increase  in  consumer  prices  was  seven  percent.  Hospital  prices 
were  increasing  at  an  annual  rate  of  eleven  percent. 

If  Medicare  expenditures  would  otherwise  increase  merely  at 
the  rate  of  general  inflation,  PSROs  would  require  benefit-cost 
ratios  in  excess  of  nine  to  one  to  completely  contain  the  rise 
in  expenditures.  This  would  imply  an  aggregate  reduction  in  days 
of  care  of  about  fourteen  percent.  Given  the  actual  (eleven  per- 
cent) increase  in  hospital  prices,  PSROs  would  require  benefit-cost 
ratios  of  nearly  fifteen  to  one,  corresponding  to  a  reduction  in 
utilization  by  almost  twenty-two  percent. 

The  benefit-cost  ratios  estimated  as  essential  for  complete 
cost  containment  would  be  extraordinary  if  observed  in  any  other 
nationwide  program.  They  exceed  the  returns  seen  in  other  govern- 
ment programs  and  in  the  private  sector.  It  would  be  unreasonable, 
therefore,  to  expect  the  PSRO  program  alone  to  be  able  to  control 
Federal  health  costs. 

A  benefit-cost  ratio  of  1.1,  however,  is  not  insignificant. 
If  it  can  be  maintained,  and,  more  importantly,  if  this  level 
of  efficiency  can  be  made  more  uniform  across  PSROs,  then  the 
program  will  be  able  to  contribute  to  a  comprehensive  cost-control 
strategy. 
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Appendix  1  to  Chapter  4 

Disaggregated  Analyses 

The  following  tables  present  the  disaggregated  results  of 
the  analyses  of  days  of  care  impacts,  expenditure  impacts,  and 
the  benefit-cost  comparisons.     The  entries  are  as  follows: 

a .  Percentage  reduction  in  days  of  care  per  thousand  -  - 
The  entries  are  estimates  of  the  percentage  difference 
between  Medicare  days  of  care  per  1,000  beneficiaries 
observed  in  calendar  year  1977  and  the  adjusted  pro- 
jection of  days  of  care  per  thousand  that  would  have 
been  observed  had  it  not  been  for  the  efforts  of  the 
PSRO. 

Estimates  of  changes  in  Medicare  days  of  care  come  from  a 
model  of  hospital  utilization  which  incorporated  data  on 
demographic  and  health  resources  characteristics  from 
virtually  all  of  the  PSRO  areas  in  the  country.  A  sta- 
tistically significant  1.5  percent  reduction  in  days  of 
care  was  found  in  the  96  PSRO  areas  which  had  implemented 
review  in  at  least  one  hospital  as  of  April  1977. 

Unfortunately,  the  model  could  not  explain  all  of  the 
variations  in  Medicare  utilization  across  PSRO  areas. 
An  adjustment  to  correct  for  this  has  been  made  in  each 
of  the  estimates,  but  some  random,  unattributable  error 
may  remain.  At  an  aggregate  level  these  errors  tend 
to  average  out.  Hence,  the  national  estimate  and  the 
census  region  aggregate  estimates  are  believed  to  be 
accurate.  Estimates  at  the  level  of  an  HEW  region  should 
also  be  accurate  where  the  estimate  is  based  on  five  or 
more  PSROs.     (Only  Region  VI  had  fewer  than  five.) 

Individual  estimates  and  the  rankings  based  on  those 
estimates  should  be  used  with  caution.  Relatively  small 
impact  differences  between  PSROs  may  not  be  significant. 
Further,  because  the  majority  of  PSRO  impact  estimates 
fall  between  plus  and  minus  2  percent,  considerable  dif- 
ferences in  ranks  between  individual  PSROs  may  also  be 
nonsignificant.  Such  differences  may  be  used  probabilis- 
tically, i.e.,  the  greater  the  difference  in  rank,  the 
greater  the  probability  that  there  be  a  true  difference. 
However,  the  exact  probabilities  cannot  be  estimated 
at  this  time. 

b.  Total  days  saved  —  The  impact  estimates  from  the  utili- 
zation model  were  multiplied  by  the  estimated  Medicare 
eligible  population  for  each  area.     The  estimates  involved 
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adjustments  for  in-migration  and  out-migration  between  each 
of  the  PSRO  areas  observed  in  1977.  (The  utilization 
analyses  incorporated  the  same  migration-adjusted  data.) 

c.  Value  of  a  day  saved  --  A  simulation  analysis  was  con- 
ducted to  assess  the  effect  on  Medicare  remi burs erne nts  of 
changes  in  Medicare  days  of  care.  A  file  of  audited 
Medicare  cost  reports  from  5,200  hospitals  was  used  in  this 
analysis.  The  simulation  took  into  account  hospitals' 
fixed  and  variable  costs,  substitutions  of  ambulatory  care 
and  long-term  care,  variations  in  the  Medicare  share  of 
hospitals'  annual  workload,  and  the  non-linear  relation 
between  changes  in  Medicare  days  of  care  and  the  Medicare 
apportionment. 

The  average  value  of  a  day  saved  was  $60.67  or  36  percent 
of  the  average  per  diem.  Variations  across  PSROs  in 
the  estimated  value  of  a  day  saved  as  a  percentage  of  per 
diem  are  due  to  variations  in  the  following  factor: 
estimated  PSRO  impact  from  the  utilization  analysis, 
Medicare  days  of  care  as  percentage  of  all  days  of  care  in 
the  sample  hospitals,  ancillary  per  diems  as  a  percentage 
of  average  short-stay  hospital  per  diems,  long-term  care 
per  diems  as  a  percentage  of  short-stay  hospital  per  diems, 
and  differences  in  occupancy  rates  between  the  sample 
hospitals  in  each  PSRO  area. 

d.  Benefit-cost  ratio  ( B/C )  —  Concurrent  review  benefits 
for  each  PSRO  area  for  CY  1977  were  estimated  as  the 
product  of  total  days  saved  and  the  value  of  a  day  saved. 
Total  concurrent  review  costs  for  Medicare  only  were 
estimated  using  adjusted  FY  1977  PMIS  data  from  each 
PSRO  area. 

Costs  included  were:  direct  cost  of  concurrent  review, 
a  fraction  of  areawide  review  costs  depending  on  rela- 
tive .expenditures  for  concurrent  review  and  MCEs,  and 
management  and  support  costs  attributed  to  concurrent 
review.  Medicare  discharges  as  a  percentage  of  all  re- 
viewed Federal  discharges  were  used  to  estimate  the  Medi- 
care costs  of  concurrent  review  for  FY  1977.  These  data 
were  then  adjusted  to  reflect  both  inflation  and  the 
increase  in  discharges  subject  to  review  in  the  fourth 
quarter  of  CY  1977. 

The  benefit-cost  estimate  is  simply  the  ratio  of  the 
benefit  and  cost  estimates.  The  aggregate  benefit-cost 
ratio  for  the  96  areas  studied  was  1.10. 
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PSRO  CONCURRENT  REVIEW  IMPACT  ON  MEDICARE  DAYS  OF  CARE 
IN  CALENDAR  YEAR  1977 


%  Reduction 

in  Days  of 
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Percent 

Care  Per 
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of  Day 

of  Per 
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Thousand 

Rank 
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Rank 
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54.85 

33 

2.67 

6 
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02  Genessee 

4.12 

17 

18,070 

8 

54.68 

35 

3.82 

4 

04  Five  County 

•  J  I 

DJ 

/b 

JO  •  4b 

"J  1 

O  1 

■  U4 

"7R 
/  O 

05  Adirondack 

(.14)* 

83 

(323) 

84 

26.87 

25 

(.04) 

83 

09  Area  9 

3.52 

21 

16,442 

10 

54.22 

33 

1  .88 

17 

10  Rockland 

5 .34 

10 

4, 126 

43 

67 .62 

40 

2.17 

14 

11  New  York 

6.53 

3 

114,785 

2 

85.69 

38 

10.69 

2 

12  Richmond 

.22 

70 

300 

73 

60.91 

37 

.19 

60 

13  Kings 

5.66 

7 

65,340 

3 

69.07 

38 

11.94 

1 

14  Queens 

1.44 

35 

12,134 

19 

44.97 

30 

2.49 

10 

15  Nassau 

.37 

60 

2,174 

52 

73.44 

38 

.42 

48 

16  Bronx 

6.42 

4 

35,971 

6 

61.29 

38 

5.74 

3 

REGION  II  TOTAL 

3.237 

283,036 

4.076 

•Values  in  parentheses  indicate  areas  which  had  increases  in  days  of  care  rather  than 
reductions.     Hence  these  areas  have  negative  benefit-cost  ratios. 
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PSRO  CONCURRENT  REVIEW  IMPACT  ON  MEDICARE  DAYS  OF  CARE 
IN  CALENDAR  YEAR  1977 


%  Reduction 

in  Days  of 

Total 

Value 

Percent 

Care  Per 

Days 

of  Day 

of  Per 

Region  III 

Thousand 

Rank 

Saved 

Rank 

Saved 

Diem 

B/C 

Rank 

DC  01 

National  Capital 

1.39  ; 

36 

4,702 

42 

$  99.32 

51 

.93 

31 

DE  01 

De laware 

.65 

53 

1 , 24  1 

65 

67 . 27 

42 

.  54 

43 

MD  01 

Western  MD 

1.48 

33 

1 ,982 

55 

44.24 

36 

.55 

41 

02 

Baltimore  City 

(.55)* 

91 

(2,575) 

91 

84.58 

42 

(.38) 

92 

03 

Montgomery  County 

5 . 37 

9 

7 , 092 

32 

83  .25 

46 

3  .75 

5 

04 

Prince  Georges 

County 

.22 

70 

130 

77 

93.11 

49 

.09 

71 

05 

Central  MD 

4.06 

19 

8,694 

23 

68.06 

43 

1 .91 

16 

07 

Delmarva 

2.63 

23 

2,776 

49 

52.83 

37 

.78 

35 

PA  02 

Central  PA 

(  .82) 

92 

(2,560) 

90 

40.97 

34 

(.33) 

90 

04 

Eastern  PA 

.  1 0 

74 

276 

74 

47  .46 

35 

.09 

71 

06 

Allegheny 

(  .43) 

89 

(394) 

85 

58.91 

38 

(.10) 

86 

07 

Southwest  PA 

1 .36 

38 

5,574 

41 

42.66 

36 

.55 

41 

08 

Highlands 

(.30) 

87 

(720) 

86 

41 .80 

33 

(.07) 

84 

09 

Southcentral  PA 

1.56 

32 

12,267 

18 

41.92 

34 

.51 

44 

11 

Mont  gome  r y /Buck  s 

4.56 

15 

18,810 

7 

51.43 

38 

1 .87 

18 

12 

Philade lphia 

1 .46 

34 

15,414 

12 

57.24 

34 

1 .06 

29 

VA  02 

Northern  VA 

.06 

78 

108 

78 

81.12 

47 

.06 

76 

05 

Colonial 

.34 

61 

1 ,347 

62 

45. 10 

41 

.18 

63 

WV  01 

West  Virginia 

.32 

62 

2,967 

47 

36.79 

36 

.12 

68 

REGION 

III  Total 

1 .153 

77,154 

.573 

Values  in  parentheses  indicate  areas  which  had  increases  in  days  of  care  rather  than 
reductions.     Hence  these  areas  have  negative  benefit-cost  ratios. 
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PSRO  CONCURRENT  REVIEW  IMPACT  ON  MEDICARE  DAYS  OF  CARE 
IN  CALENDAR  YEAR  1977 


%  Reduction 

in  Days  of 

Total 

Value 

Percent 

Care  Per 

Days 

of  Day 

of  Per 

Region  IV 

Thousand 

Rank 

Saved 

Rank 

Saved 

Diem 

B/C 

Rank 

AL  01  Alabama 

( .22)* 

85 

3,084 

93 

$  42.76 

36 

( .  16) 

88 

FL  12  Dade-Monroe 

1.23 

41 

12,572 

16 

41.33 

21 

.62 

39 

Kl  u l  Kentucky 

fob 

J  ,  /  Jo 

A  A 

44 

1C  Oft 

1  A 

34 

.21 

59 

MS  01  Mississippi 

(1.15) 

94 

( 12,235) 

94 

23.75 

25 

(.21) 

89 

SC  0 1  South  Carolina 

r  4C    ft  -a  o  \ 
(ID,  U 6Z ) 

jy  •  /  j 

Jo 

(  .  70 ) 

95 

TN  02  Tennessee  FMC 

(1.77) 

95 

(25,324) 

96 

37.78 

33 

(.48) 

93 

REGION  IV  TOTAL 

( .577) 

( 39 , 365 ) 

( .202) 

REGION  V 



IL  03  Chicago  FMC 

.56 

56 

13,604 

14 

$  69.31 

39 

1.78 

22 

04  Quad  River 

4.63 

13 

6,723 

34 

55.86 

44 

2.09 

15 

MI  0 1  Upper  Peninsula 

1.27 

39 

2,163 

53 

35.04 

29 

.17 

64 

05  GLSC 

.05 

79 

107 

79 

65.73 

45 

.  02 

79 

MN  01  FHCE 

1.02 

45 

8,636 

25 

54.15 

37 

1.01 

30 

02  PSQC 

.31 

63 

1 ,587 

59 

21 .86 

20 

.37 

49 

OH  01  MEDCO 

.31 

63 

1,931 

57 

54.67 

39 

.19 

60 

02  Region  II 

.22 

70 

833 

6 

57.16 

41 

.16 

66 

04  Fourth  Ohio 

.57 

55 

2,048 

54 

65.60 

42 

.27 

56 

10  Peer  Review 

Systems 

.44 

57 

1,982 

55 

56.56 

39 

.26 

57 

12  Physicians  Peer 

Review 

(.36)* 

88 

(3,055) 

92 

74.54 

47 

(.51) 

94 

WI  01  Wisconsin 

.10 

74 

1,297 

64 

40.50 

32 

.05 

77 

02  SE  Wisconsin 

1.77 

30 

13,120 

15 

65.04 

39 

1.32 

27 

REGION  V  Total 

.436 

50,976 

.495 

♦Values  in  parentheses  indicate  areas  which  had  increases  in  days  of  care  rather  than 
reductions.     Hence  these  areas  have  negative  benefit-cost  ratios. 
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PSRO  CONCURRENT  REVIEW  IMPACT  ON  MEDICARE  DAYS  OF  CARE 
IN  CALENDAR  YEAR  1977 


Region  VI 

■%  Reduction 
in  Days  of 
Care  Per 
Thousand 

Rank 

Total 
Days 
Saved 

Rank 

Value 
of  Day 
Saved 

Percent 
of  Per 
Diem 

B/C 

Rank 

AR  0 1  Arkansas 

.66 

52 

7,150 

30 

$  24.84 

24 

.17 

64 

NM  01  New  Mexico 

2.61 

24 

7,196 

29 

51.84 

36 

.80 

33 

REGION  VI  TOTAL 

1.139 

14,346 

.372 

R^cri  on  X7TT 

IA  01  Iowa 

.58 

54 

8,539 

26 

$  39.88 

32 

.45 

46 

MO  0 1  Northwest  MO 

02  Mid  MO 

03  CEMPROC 

04  MOAF 

05  Southeast  MO 

( .  15)* 
1.01 
.10 
(.26) 
1.24 

84 
46 
74 
86 
40 

f 1 . 1221 
3,696 
1,012 

(892) 
3,006 

88 
45 
66 
87 
46 

38 .60 
29.78 
45.22 
19.97 
23.97 

29 
24 
33 
20 
23 

( .  14) 

V.I**/ 

.28 
.07 
(.07) 
.32 

87 
55 
73 
84 
52 

REGION  VII  TOTAL 

.340 

14,239 

.192 

Region  VIII 

CO  01  Colorado 

.87 

51 

7,142 

31 

$  56.14 

37 

.35 

51 

MN  01  Montana 

.94 

4b 

2,488 

51 

47.44 

36 

.19 

60 

ND  01  North  Dakota 

(.04)* 

82 

(140) 

83 

38.98 

30 

(.01) 

82 

SD  01  South  Dakota 

.43 

58 

1,438 

61 

30.50 

28 

.15 

67 

UT  01  Utah 

4.63 

13 

12,374 

17 

63.86 

47 

2.49 

23 

WY  01  Wyoming 

8.75 

1 

9,510 

21 

49.93 

41 

1.71 

23 

REGION  VIII  TOTAL 

1.712 

32,812 

.567 

' *Values  in  parentheses  indicate  areas  which  had  increases  in  days  of  care  rather  than 
reductions.     Hence  these  areas  have  negative  benefit-cost  ratios. 
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PSRO  CONCURRENT  REVIEW  IMPACT  ON  MEDICARE  DAYS  OF  CARE 
IN  CALENDAR  YEAR  1977 


%  Reduction 

in  Days  of 

Total 

Value 

Percent 

Care  Per 

Days 

of  Day 

of  Per 

Region  IX 

Thousand 

Rank 

Saved 

Rank 

Saved 

Diem 

B/C 

Rank 

CA  01  Redwood  Coast 

6.10 

5 

7,381 

28 

$  69.41 

33 

1  .81 

19 

03  Northbay 

5.00 

12 

5,952 

39 

78.40 

34 

1.79 

21 

04  Greater 

Sacramento 

1.17 

42 

2,810 

48 

85.40 

37 

.51 

44 

05  San  Francisco 

1.77 

30 

6,308 

35 

87.17 

34 

.59 

40 

08  San  Joaquin 

1.39 

36 

1 ,341 

63 

72.14 

38 

.72 

37 

09  Santa  Clara 

.29 

66 

690 

71 

109.87 

45 

.25 

58 

1 2  Monterey 

7.49 

2 

7,757 

27 

67.54 

37 

2.29 

13 

14  Kern 

.28 

67 

245 

75 

68.95 

33 

.07 

73 

16  Santa  Barbara 

(1.01)* 

93 

(1/342) 

89 

60.23 

33 

( 1 .36) 

96 

17  Ventura 

.97 

47 

824 

70 

82.75 

41 

.30 

54 

18  L.A.  Combined 

2.32 

27 

54,404 

4 

70.92 

31 

2.56 

7 

27  Riverside 

5.71 

6 

11,178 

20 

49.78 

26 

1 .65 

24 

HA  01  Pacific 

.92 

50 

1,615 

58 

75.87 

38 

.36 

50 

NV  01  Nevada 

.01 

81 

17 

81 

79.44 

43 

.004 

81 

REGION  IX  TOTAL 

2.249 

99,180 

1.234 

Region  X 

AK  01  Alaska 

(.53)* 

90 

(104) 

82 

$148.95 

57 

(.36) 

91 

ID  01  Idaho 

2.58 

26 

6,291 

36 

51.70 

37 

.91 

32 

OR  01  Multnomah 

5.10 

11 

14,914 

13 

76.30 

39 

2.56 

7 

02  Greater  Oregon 

1.14 

43 

6,090 

38 

57.63 

35 

.65 

38 

WA  01  Washington 

4.34 

16 

43,177 

5 

64.^4 

36 

2.52 

9 

REGION  X  TOTAL 

3.087 

70,368 

1.843 

♦Values  in  parentheses  indicate  areas  which  had  increases  in  days  of  care  rather  than 
reductions.    Hence  these  areas  have  negative  benefit-cost  ratios. 
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5.      QUALITY  ASSURANCE  ACTIVITIES:     MEDICAL  CARE  EVALUATION  STUDIES 


5.1  Overview 


Throughout  the  legislative  history  of  PSRO,  the  theme  of 
utilization  control  has  been  paralleled  by  the  concern  that  the 
services  delivered  meet  professional  standards  of  quality.  While 
opinions  have  varied  regarding  the  relative  emphases  on  these  two 
aspects  of  the  program,  it  has  always  been  recognized  that  the 
reduction  of  utilization  rates  at  the  expense  of  the  quality  of  care 
provided  to  Medicare,  Medicaid  and  Maternal  and  Child  Health  bene- 
ficiaries would  be  unacceptable.  Further,  the  PSRO  Act  implies  that 
quality  of  care  should  be  not  only  maintained  but,  if  possible, 
improved.  This  section  of  the  evaluation  report  deals  with  the  PSRO 
program's  current  principal  mechanism  for  monitoring,  assuring, 
and  improving  the  quality  of  medical  care:  the  medical  care  evalua- 
tion studies  (MCE)  component. 

5.1.1  Ob j  ectives 

Two  issues  are  paramount  in  assessing  the  MCE  component  of  the 
PSRO  program:  implementation  and  impact .  This  section  reports  two 
studies:  the  first  (Section  5.2)  summarizes  the  extent  to  which 
MCEs  have  been  implemented  to  date;  the  second  (Section  5.3)  is 
concerned  with  the  initial  development  of  a  research  strategy  for 
measuring  program  impact  on  the  quality  of  care.  A  further  study  of 
MCE  impact,  based  on  the  methodology  of  the  second  study  but  with 
considerable  refinements  in  sampling  and  data  collection,  is  sched- 
uled for  completion  in  1979. 

5.1.2  Studies  and  Relationships 

The  first  study,  "Development  and  Current  Status  of  MCE  Activ- 
ity," reports  the  compliance  of  PSROs  with  the  HSQB  requirements 
regarding  MCEs,  assessed  through  analysis  of  PSRO  Management  Infor- 
mation System  (PMIS)  data.  Numbers  of  initial  audits  are  reported 
over  time/  for  individual  PSROs,  and  by  topic  categories.  Reaudits 
are  also  reported  over  time. 

The  second  study,  "Pilot  Variation  Rate  Analysis,"  develops  a 
methodology  and  presents  preliminary  findings  regarding  the  impact 
of  MCEs  on  the  quality  of  patient  care.  It  represents  a  pilot 
application  of  a  method  for  measuring  the  degree  of  deviation  from 
standards  of  good  care  —  to  be  referred  to  as  the  "variation  rate" 
—  before  and  after  MCEs  are  performed. /1 


/I  The  variation  rate,  discussed  in  Section  5.3.3,  is  defined  as 
the  proportion  of  patient  records  which  failed  to  meet  the  stan- 
dard established  for  a  given  criterion.  Variation  rates  were 
measured  both  at  audit  and  reaudit;  any  differences  noted  were 
taken  to  be  a  measure  of  change. 
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To  understand  the  strengths  and  limitations  of  the  pilot  study, 
it  is  desirable  to  view  it  in  the  context  of  a  strategy  for  MCE 
assessment.  The  ultimate  goal  in  the  assessment  of  MCE  studies 
parallels  that  of  any  program  impact  evaluation:  to  measure  changes 
in  quality  of  care  within  a  research  design  which  can  reasonably 
attribute  change  to  the  influence  of  the  PSRO  program.  The  impact 
study  reported  here  is  developmental  in  the  following  sense: 
although  it  presents  a  method  of  measuring  changes  in  the  quality  of 
care,  it  makes  use  of  neither  a  representative  sample  of  MCEs. nor 
comparison  data  from  hospitals  which  have  not  performed  MCEs.  Based 
on  the  experience  gathered  in  this  pilot  study,  a  larger  study 
currently  underway  will  overcome  the  first  of  these  limitations:  it 
will  examine  changes  in  compliance  to  standards  of  care  among  a 
nationally-representative  sample  of  1,000  MCEs.  A  third  step  in  the 
overall  assessment  strategy  will  be  to  collect  data  which  will  allow 
quasi-experimental  comparisons,  either  by  measuring  changes  in 
compliance  to  standards  at  sites  where  MCEs  have  not  been  performed, 
or  by  generating  a  time  series  of  measurements  at  MCE  sites. 

Compared  to  the  fully-implemented  evaluation  strategy  just 
described,  the  pilot  impact  study  reported  in  this  section  is 
incomplete.  Instead  of  a  statistically  representative  sample  of 
approximately  1,000  MCEs,  the  pilot  sample  includes  only  136  MCEs, 
solicited  from  those  PSROs  known  to  have  completed  a  large  number 
of  reaudits  in  selected  topic  categories.  Data  on  both  "before"  and 
"after"  compliance  with  criteria  came  from  PSRO  or  hospital  records, 
with  little  control  of  data  quality.  No  comparison  data  were 
collected. 

Given  these  constraints  on  the  validity  of  the  pilot  study,  it 
is  important  to  emphasize  what  can  be  gained  from  such  a  limited 
design: 

•  It  permits  a  preliminary  test  of  the  Variation  Rate  Method- 
ology. 

•  It  provides  a  detailed  look  at  how  MCEs  are  performed, 
particularly  a  clinical  assessment  of  how  criteria  vary  from 
study  to  study  within  a  topic  area. 

•  It  provides   a   "first    look"   assessment   of   program  impact. 

While  the  MCEs  examined  in  the  pilot  study  do  not  constitute  a 
random  sample  and  thus  cannot  represent  the  universe  of  MCE  studies, 
these  particular  studies  came  from  those  PSROs  which,  at  an  early 
stage  of  program  development,  have  been  most  effective  in  encourag- 
ing compliance  with  the  requirement  that  reaudits  be  done.  These 
PSROs  have  also  had  the  most  experience  in  MCE,  as  measured  by 
the  numbers  of  initial  audits  completed  (see  Section  5.3.2  on  how 
PSROs  were  selected  for  this  study). 
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5.1.3    Data  Sources 


The  studies  reported  in  this  section  use  most  of  the  MCE  data 
which  are  part  of  the  PSRO  Management  Information  System  (PMIS). 
PMIS  data  are  used  extensively  in  the  MCE  activity  study;  the 
variation  rate  analysis  employed  in  the  impact  study  uses  PMIS  data 
mainly  for  sampling.  Further  information  needed  for  each  study  was 
solicited  directly  from  the  PSROs.  This  subsection  will  review 
these  data  sources  with  an  emphasis  on  their  potential  usefulness 
for  program  evaluation.  It  is  recognized  that  where  critical  gaps 
exist  in  available  data,  information  must  be  specially  collected  to 
fill  them.  No  such  primary  data  collection  was  conducted,  however, 
for  the  studies  reported  here. 

The  MCE  activity  study  relies  on  data  from  routine  reports 
which  the  PSROs  submit  to  the  PMIS  on  the  specific  characteristics 
of  completed  audits  (Form  HSQB  131)  and  reaudits  (Form  HSQB  133), 
plus  quarterly  summaries  of  all  MCE  studies  completed  and  in  process 
(Form  HSQB  135).  The  forms  received  by  the  PMIS  contractor  are 
checked  for  completeness  and  internal  consistency;  those  which  pass 
this  minimal  edit  are  entered  into  the  computer  data  base,  while 
others  are  returned  to  the  PSROs  for  correction. 

PMIS  data  on  MCEs  are  stored  in  a  format  specially  adapted  to 
the  purpose  of  providing  technical  assistance  to  PSROs  through 
the  MCE  Clearinghouse. /A_  Although  programs  to  perform  a  limited 
number  of  analyses  are  part  of  the  Clearinghouse,  these  must  be 
supplemented  by  hand  tabulations  and  additional  programming  for 
evaluative  purposes.  Indeed,  all  activity  data  for  1978  included  in 
this  report  are  the  products  of  hand-tabulations,  necessitated 
because  of  constraints  in  available  time  and  personnel  and  diffi- 
culties in  data  processing. 

The  quality  of  PMIS  data  on  MCEs  has  yet  to  be  systematically 
assessed.  It  is  hoped  that  the  next  report  in  the  sequence  of  MCE 
evaluations  (available  in  1979)  will  include  such  an  assessment. 
Until  then,  the  evaluation  will  rely  on  judgments  by  informed  HSQB 
personnel  regarding  time  lags  in  reporting  and  validity  of  items 
reported.  Data  on  reaudits  (other  than  the  numbers  reported  each 
quarter)  and  on  areawide  audits  will  be  reported  when  their  validity 
has  been  established. 


/ 1  The  MCE  Clearinghouse  is  described  in  the  MCE  Clearinghouse 
Quarterly,  Number  1-78  (July  1978),  published  by  the  Division 
of  Peer  Review,  OPSRO,  HSQB,  HCFA. 
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In  contrast  to  the  activity  study,  the  impact  (or  variation 
rate)  study  required  more  detailed  data.  After  a  sample  of  MCEs  was 
selected  through  the  PMIS,  further  information  on  each  study  was 
requested  from  the  PSRO  involved.  A  variety  of  background  documen- 
tation, not  provided  by  the  PMIS,  is  ordinarily  available  for 
research  purposes  from  PSROs.  For  example,  audit  committee  minutes 
may  document  the  process  of  selection  of  the  MCE  topic  and  criteria; 
standards,  sample  sizes,  findings  and  justifications  are  usually 
recorded  in  a  standard  audit  form,  such  as  the  JCAH  PEP  form; 
finally,  the  analysis  of  deficiencies,  recommendations  for  correc- 
tive action,  and  planning  for  reaudit  may  be  found  in  still  another 
set  of  committee  minutes  and  memoranda.  Audit  forms  constituted  the 
primary  data  source  for  the  variation  rate  analysis. 

It  might  be  noted  that  the  willingness  of  hospitals  and  PSFOs 
to  provide  such  data  is  an  important  factor:  they  are  not  required 
to  participate  in  program  evaluation  and  may  be  reluctant  to  do  so 
because  of  a  concern  for  confidentiality,  limitations  of  personnel, 
or  simply  irritation  at  frequent  Federal  requests.  Experience  has 
shown,  however,  a  gratifyingly  high  rate  of  cooperation. 

The  quality  of  the  information  obtained  from  the  hospitals 
and  PSROs  varies.  This  does  not  mean  that  the  records  are  defi- 
cient; they  are  no  doubt  adequate  to  the  task  for  which  they  were 
intended,  i.e.,  the  actual  performance  of  an  audit  and  reaudit.  For 
evaluation  purposes,  however,  they  may  not  contain  the  information 
which  is  sought,  or  may  not  contain  this  information  consistently 
across  studies.  Part  of  the  pilot  variation  rate  analysis,  for 
example,  consists  of  a  close  examination  of  the  clinical  signifi- 
cance of  observed  variations  from  standards  of  care  and  the  changes 
in  variation  rates  found  from  audit  to  reaudit.  For  such  "micro- 
scopic" examinations,  as  many  original  documents  as  possible  were 
obtained  through  the  PSROs.  Even  with  this  supplemental  material, 
evaluators  had  difficulty  at  times  ascertaining,  for  example,  why 
certain  criteria  were  chosen  to  be  audited  and  reaudited,  and  at 
times  why  reaudits  were  done  at  all.  The  acquisition  of  systematic 
information  on  these  and  other  details  of  the  MCE  process  would 
require  special  data  collection  efforts. 

5.1.4    Relationship  to  the  1977  MCE  Evaluation 

This  year's  evaluation  of  the  MCE  component  of  the  PSRO  program 
builds  on  last  year's  analysis,  but  includes  both  methodological 
advances  and  a  shift  in  emphasis  from  the  MCE  process  to  its  impact. 
The  greatest  increment  of  the  studies  presented  here  is  in  the  area 
of  MCE  impact  analysis.  The  1977  report  included  an  analysis  of  the 
changes  in  medical  care  brought  about  by  37  MCE  audits/reaudits  in 
four  topic  categories.  This  year's  report  includes  not  only  a 
larger  sample  of  studies  in  a  greater  range  of  topics,  but  takes  a 
more  thorough  look  at  changes  in  variation  rates,  both  statistically 
and  clinically. 
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5.2     Study;   Development  and  Current  Status  of  MCE  Activity 


5.2.1  Objectives 

This  section  describes  the  implementation  of  the  MCE  component 
of  the  PSRO  program  in  terms  of  the  number  of  studies  completed  from 
the  last  quarter  of  1975  through  the  second  quarter  of  1978  and  the 
distribution  of  those  studies  (a)  over  PSROs  and  (b)  among  topic 
categories.  The  characteristics  of  the  MCE  cycle  are  specified  in 
PSRO  Transmittal  No.  43/1  and  the  Handbook  for  the  Conduct  of  Med- 
ical Care  Evaluation  Studies. /2 

The  basic  mechanism  of  the  MCE  is  an  adaptation  of  the  medical 
audit,  a  method  of  peer  review  which  has  gained  increasing  accept- 
ance in  recent  decades.  Generally,  medical  audits  involve  a  series 
of  steps:  specification  of  a  topic,  sampling  of  records,  establish- 
ment of  criteria  of  good  care,  comparison  of  criteria  with  actual 
performance,  analysis  of  findings,  and  recommendation  of  action  to 
correct  the  deficiencies  detected.  The  significant  advances  of  the 
PSRO  program  over  traditional  medical  audits  are:  (1)  reaudits  to 
confirm  the  effectiveness  of  recommended  actions  have  been  made  a 
routine  part  of  the  MCE  cycle;  (2)  PSROs  have  been  made  responsible 
for  the  number  and  quality  of  MCEs  performed,  either  conducting  the 
studies  themselves  or  delegating  the  task  to  qualified  hospitals; 
(3)  area-wide  (multi-site)  MCEs  have  been  initiated;  and  (4)  MCE 
program  monitoring  and  technical  assistance  have  become  the  func- 
tions of  a  central  office. 

Data  will  be  reported  separately  for  MCE  audits  (the  initial 
step  of  the  process)  as  well  as  reaudits  (the  final  step  in  which 
the  quality  of  care  is  re-evaluated  following  the  taking  of  correc- 
tive action) . 

5.2.2  Data 

Information  on  MCE  activity  has  been  gathered  through  the  PMIS 
since  the  beginning  of  the  program  in  1975.  Reporting  by  the  PSROs 
is  quarterly  through  the  use  of  Forms  HSQB  131,  133  and  135.  The 
first  two  of  these  forms  were  used  to  provide  data  for  this  analy- 
sis. 

PMIS  data  are  maintained  by  HSQB  in  computerized  form.  Routine 
reports  generated  for  the  MCE  Clearinghouse  were  the  source  of  most 
of  the  data  used  in  the  analysis,  with  special  tabulations  and 
rankings  prepared  by  study  staff. 

No  attempt  was  made  to  validate  the  data  elements  accumulated 
through  PMIS.     As  noted  previously,   however,   forms  received  by  the 


/1  DHEW/HSA/BQA,  PSRO  Transmittal  No.  43,  January  25,  1977. 
/2  DHEW/HCFA/HSQB,   1978,  Jeffrey  A.  Perlman,  M.D.  Editor. 


186 


PMIS  data  contractor  are  checked  for  completeness  and  consistency; 
problems  are  clarified  by  telephone  if  possible  —  otherwise  the 
forms  are  returned  to  the  PSRO  for  correction.  The  quality  of  the 
data  for  this  study  is  further  influenced  by  several  time-dependent 
factors : 

1 .  Delays  in  reporting  completed  studies*  —  It  is  required, 
under  the  PSRO  program,  to  meet  minimum  requirements 
for  number  of  completed  MCE  studies.  To  date,  however, 
studies  have  often  not  been  reported  within  the  allowed 
45-day  reporting  period  following  the  quarter  in  which  the 
study  was  completed. 

2.  Time  lags  due  to  data  processing.  —  There  are  sometimes 
lags    of   several  months  in  the  data  processing  cycle. 

3 .  Delays  due  to  time  necessary  for  resolution  of  problems  in 
submitted  PMIS  forms.  —  Forms  returned  to  PSROs  to 
resolve  problems  of  incompleteness  or  inconsistency  can 
result  in  further  delays  of  one  to  two  months. 

Most  of  the  tables  presented  in  this  report  are  based  on  data 
available  as  of  October  6,  1978.  It  is  recognized  that  some  MCEs 
completed  in  the  most  recent  quarters  may  be  excluded.  In  some 
instances,  to  be  identified  in  the  text,  an  effort  was  made  to 
obtain  more  complete  data  for  the  first  and  second  quarters  of  1978 
through  hand-tabulation. 

5.2.3  Methods 

MCE  topics  are  selected  by  committees  of  physicians  as  an 
aspect  of  the  MCE  design  process,  and  may  be  narrowly  specified. 
For  purposes  of  this  analysis,  topics  were  grouped  into  578  topic 
categories  judged  to  preserve  the  most  meaningful  distinctions  among 
topics. /_1  These  topic  categories  generally  represent  medical 
diagnoses  or  surgical  and  administrative  procedures. 

For  descriptive  purposes  in  some  of  the  figures  which  follow, 
MCE  topics  have  been  further  grouped  into  three  major  topic  classi- 
fications: medical,  surgical,  and  administrative  and  miscellaneous 
procedures.  Through  expert  consensus,  each  topic  category  was 
placed  into  one  of  these  three  classifications,  which  are  defined  as 
follows: 

•        Classification  A:  Medical 

This  classification  includes  all  medical  diagnoses  and 
procedures  for  which  the  generally  accepted  treatment  is 
medical.  Gastric  ulcers,  leukemias,  lymphomas  and  neo- 
plasms not  treated  surgically  were  included  in  this 
classification. 


/1  Coding  was  carried  out  according  to  Equivalence  Tables  5  and  6, 
published  by  the  Commission  on  Professional  and  Hospital  Activi- 
ties, 1974. 
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Classification  B:  Surgical 


This  classification  includes  all  surgical  and  obstetrical 
procedures  as  well  as  invasive  diagnostic  procedures.  It 
also  includes  those  diagnoses  for  which  surgery  is  gener- 
ally indicated,  such  as  breast  lesions  and  cholelithia- 
sis. Neoplasms  usually  treated  surgically  were  included 
in  this  classification. 

•       Classification  C:     Administrative  and  Miscellaneous 

Procedures 

This  classification  includes  administrative  topics  such  as 
blood  usage  or  length  of  stay.  Studies  on  the  use  of 
particular  facilities  or  services  within  the  hospital, 
such  as  emergency  room  and  x-rays,  were  coded  into  Class- 
ification C. 

Data  are  generally  reported  by  quarter,  but  in  some  tables 
calendar  years  are  used  for  greater  clarity.  In  all  cases,  the  time 
reported  is  the  date  that  the  MCE  (audit  or  reaudit)  was  completed 
by  the  PSRO  or  hospital  rather  than  the  date  it  was  entered  into  the 
data  base. 

5.2.4    Quantity  and  Distribution  of  the  MCE  Activity 

Figure  6  shows  the  total  number  of  initial  MCE  audits  report- 
ed through  the  PMIS  since  early  1975.  The  number  of  audits  rises 
steadily  until  the  third  quarter  of  1977,  when  a  slight  decline 
occurs.  This  unexplained  decrease  contrasts  with  the  marked  in- 
creases noted  in  previous  quarters.  The  fourth  quarter  of  1977 
shows  a  continued  increase  in  MCE  activity,  and  initial  reports  of 
the  first  quarter  of  1978  further  verify  this  upward  direction.  The 
subsequent  decline  noted  in  the  second  quarter  of  1978  is  apparently 
a  function  of  reporting  delays./1 

Figure  7  presents  the  growth  in  MCE  audit  activity  both  in 
the  number  of  PSROs  reporting  audits  and  in  the  average  number  of 
audits  per  reporting  PSRO.  Further  analysis  of  the  data  used  to 
develop  this  chart  indicated  that  during  1975-76,  73  separate  PSROs 
reported  audits  as  compared  to  a  total  of  97  PSROs  reporting  audits 
during  1977.  This  growth  in  PSRO  activity  reflects  the  PSROs' 
increased  operational  capabilities.  It  should  be  noted  that  there 
is  a  start-up  time  required  for  PSROs  to  develop  the  necessary 
organizational  capacity  to  perform  review  functions  as  well  as 
to  delegate  any  or  all  of  those  functions  to  hospitals.  During 
these  early  stages  the  PSRO  directs  the  greater  portion  of  its 
administrative  attention  toward  concurrent  review.  Consequently, 
the  implementation  of  MCEs  is  a  relatively  late  development,  gener- 
ally taking  place  a  year  or  more  after  the  institution  of  concurrent 
review. 


/1  The  figures  for  1978,  indicated  by  clear  bars,  include  updated 
information  which  was  hand-tabulated.  The  data  for  these  quar- 
ters may  not  yet  be  complete. 
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FIGURE  6:   MCE  Initial  Audit  Activity,  By  Quarter 
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FIGURE  7:    Average  PSRO  MCE  Audit  Activity,  By  Quarter 
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Table  50  shows  a  wide  variation  in  MCE  audit  activity  among 
PSROs  as  of  the  second  quarter  of  1978,  ranging  from  as  few  as  one 
audit  to  as  many  as  1  ,270.  The  information  in  this  table  is  ex- 
panded upon  in  Figure  8  which  indicates  that  35%  of  all  MCE  activity 
through  1977  can  be  attributed  to  the  ten  most  active  PSROs.  Not 
shown  in  the  figure  but  evident  from  Table  50,  are  the  following 
observations : 

•  The  single  most  active  PSRO  has  accounted  for  5.6%  of 
all  reported  audits. 

•  Nine  of  the  ten  most  active  PSROs  began  MCE  audits  in 
1975-76  and  have  reached  and  maintained  a  high  level 
of  productivity  through  at  least  the  first  quarter  of 
1978./M 

•  One  of  the  ten  most  active  PSROs  did  not  begin  reporting 
until  the  second  quarter  of  1977  yet  became  the  most 
active  by  the  end  of  that  year  and  has  maintained  that 
position  since. 

Though  MCE  activity  is  concentrated  in  the  most  productive 
PSROs,  the  distribution  is  somewhat  more  even  than  that  presented  in 
the  1977  report,  in  which  the  top  ten  PSROs  accounted  for  45%  of  all 
audits.  On  the  other  hand,  Figure  8  continues  to  show  that  many 
PSROs  are  producing  few  MCEs:  the  69  least  productive  PSROs  among 
the  109  which  reported  MCEs  produced  only  25%  of  initial  audits. 
Not  all  of  these  PSROs  are  new;  some  of  the  oldest  remain  low 
producers  of  MCEs.  Only  a  few  PSROs  are  known  to  have  withheld  MCE 
reports  from  the  PMIS.  The  question  for  further  study  is  how  PSROs 
which  report  few  audits  are  fulfilling  their  responsibility  to 
monitor  and  improve  quality  of  care. 

Reaudit  frequencies,  presented  in  Figure  9,  are  considerably 
lower  than  the  figures  for  initial  audits.  Only  414  reaudit  reports 
were  found  in  the  computer  file  for  1975-77.  When  these  were 
augmented  by  hand  counts  of  reaudits  reported  for  1978  but  not  yet 
entered  in  the  computer  file,  the  total  reached  1,198. 

Some  lags  between  audit  and  reaudit  figures  may  be  expected  due 
to  the  fact  that  reaudits  are  usually  performed  up  to  one  year  after 
the  initial  audit,  but  even  allowing  for  this  lag,  the  number  of 
reaudits  is  less  than  one-fifth  of  the  number  expected. /2  PSROs 
are  thus  not  receiving  and/or  reporting  feedback  on  the  efficacy  of 
actions   taken  to  remedy  deficiencies   found  in  the  initial  audits. 


/1  Declines  noted  in  the  second  quarter  of  1978  should  be  discounted 
as  probably  attributable  to  lags  in  reporting. 

/2  This  estimate  is  based  on  the  finding  that  70  percent  of  initial 
audit  reports  (HSQB  Form  131)  indicate  that  one  or  more  unjusti- 
fied variations  were  found. 
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TABLE  50 

QUARTERLY  DISTRIBUTION  OF  MCE  AUDIT  ACTIVITY 
(Reported  by  PSROs  as  of  September  1.  1978) 
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;  19 

24 

30 

26 

27 

44 

58 

36 

282 

3PA08 

_ 

_ 

9 

7 

i  30 

32 

31 

112 

3PA09 

14 

36 

i  42 

54 

!  s 

51 

53 

85 

78 

555 

3PA11 

11 

J  7 

!  20 

I  32 

43 

39 

40 

;  36 

28 

46 

333 

3PA12 

- 

- 

3 

i  4 

21 

!  37 

54 

20 

69 

68 

23 

299 

3VA02 

_ 

i  : 

|  - 

1 

~ 

- 

9 

15 

30 

19 

73 

3VA0S 

!  : 

- 

I  15" 

1  : 

1 

1 

22 

22 

61 

45 

21 

173 

3WV01 

- 

8 

30 

37 

|  50 

70 

47 

69 

75 

46 

454 

4AL0  1 

! 

1 

1 

21 

I  60 

78 

116 

177 

189 

134 

777 

4FL03 

i 

~ 

- 

20 

« 

28 

4FL12 

i  1 

!  5 

22 

60 

45 

58 

,97! 

SO 

56 

373 

4MS0  1 

17 

28 

i  50 

i  69 

99 

148 

135 

134 

154 

142 

1270 

4NC02 

I 
1 

I 

i 

i 

i 

j  

; 

i 
i 

1 

!  5! 

1 
1 

i 

1 
1 

!  - 
i  I 

11 

12 
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TABLE  50  (Continued) 
QUARTERLY  DISTRIBUTION  OF  MCE  AUDIT  ACTIVITY 
(Reported  by  PSROe  as  of  September  1,  1978) 


QUARTERS 

I  *Da 

1975 

1975 

1975 

1976 

1976 

1976 

19  76 

1977 

1977 

1977 

1978 

1978 

NUMBER 

2 

3 

4 

2 

3 

4 

1 

2 

3 

1  

2 

4  SC0 1 

2 

10 

33 

49 

74 

72 

84 

84 

99 



72 

79 

65 



723 

4TH02 

3 

2 

15 

29 

60 

80 

112 

111 

107 

105 

115 

195 

1056 

SILOS 

6 1 

56 

212 

264 

184 

777 

5IL04 

| 

4 

g 

10 

5 

3 

7 

5 

4 

2 

17 

e 

70 

SMIO  1 

2 

4 

g 

5MI05 

2 

4 

18 

18 

17 

20 

20 

D 

14 

133 

5  MHO  1 

4 

31 

35 

5  MHO  2 

9 

22 

33 

36 

39 

55 

45 

42 

46 

72 

8  3 

508 

5MH03 

13 

1 1 

22 

47 

5OH01 

\ 

1 

3 

30 

68 

48 

46 

59 

4 1 

36 

43 

32 

408 

S0H02 

16 

23 

38 

1 1 

68 

50H04 

2 

16 

22 

23 

26 

28 

39 

25 

214 

5OR06 

E 

5 

5OH10 

14 

28 

34 

58 

2 1 

29 

184 

50H12 

i 

1 1 

3 1 

37 

1 1 

93 

5HX0 1 

61 

9 

32 

57 

104 

157 

152 

1 29 

124 

892 

5WX02 

B 

25 

10 

75 

58 

80 

48 

76 

87 

660 

6AP0 1 

5 

12 

30 

39 

99 

58 

61 

66 

43 

486 

6  HM0 1 

| 

c 

7 

19 

30 

2 1 

16 

10 

16 

10 

E 

145 

7 1  A0 1 

20 

37 

3 1 

60 

122 

1 15 

385 

7KS0  1 

14 

14 

7MO02 

6 

43 

47 

38 

75 

25 

35 

269 

7  MOO  3 

7 

9 

3 1 

37 

36 

43 

69 

48 

7  MOO  4 

5 

24 

36 

26 

34 

42 

167 

8CO0 1 

23 

36 

50 

51 

59 

45 

66 

77 

54 

46 1 

8MT0 1 

2 

3 

1 

2 

g 

8MO0 1 

20 

39 

38 

44 

63 

54 

258 

BSO0 1 

12 

1 1 

30 

28 

35 

4 1 

30 

SUT0 1 

1 

1 

1 

2 

1 

2 

5 

1 

16 

8WY0 1 

2 

2 

9 

1 

8 

2 

10 

41 

4 

80 

9CA0 1 

2 

1 

21 

8 

8 

1 1 

3  1 

18 

10 

1 1 

17 

189 

9CA03 

12 

20 

22 

10 

12 

9 

2 

18 

13 

7 

20 

11 

134 

9CA04 

3 

17 

12 

4 

4 

9CA0S 

- 

6 

13 

24 

49 

36 

41 

169 

9CA08 

10 

3 

15 

14 

1 1 

15 

1 1 

7 

110 

9CA09 

3 

8 

3 

n 

13 

17 

14 

19 

77 

9CA10 

4 

20 

24 

9CA12 

3 

10 

ii 

3 

9 

1 

39 

9CA14 

1 

1 

1 

4 

19 

13 

13 

13 

23 

43 

129 

9CA16 

3 

8 

1 

12 

9CA17 

10 

7 

9 

17 

13 

9 

65 

9CA20 

1 

13 

29 

40 

75 

79 

237 

9CA21 

4 

4 

9CA22 

3 

37 

18 

19 

12 

4 

93 

9CA23 

1 

6 

30 

15 

37 

47 

136 

9CA27 

1 

5 

10 

14 

4 

12 

18 

30 

10 

34 

13 

18 

169 

9HI01 

1 

2 

1 

37 

14 

36 

26 

117 

9HV01 

6 

23 

34 

24 

28 

18 

133 

TOTAL  8 

34 

97 

298 

533 

874 

1116 

1554 

2064 

2860 

2860 

2860 

3844 

3030 

22808 
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FIGURE  8:    Percent  of  Total  MCE  Activity,  By  Ranked  Groupings  (1976-77) 


1-10  11-20         21-30  31-40  41-50  51-60  61-70  71-80 

Rank  Groupings  of  PSROs 


81-90         91-100  100-109 
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FIGURE   9:   MCE  Reaudit  Activity,  By  Quarter 


450 
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MCE  Initial  Audit  Topics 


Major  Classification;  The  topic  of  each  individual  MCE  is 
reported  verbatim  on  the  PMIS  form*  Table  51  shows  the  distribution 
of  audits  for  the  two  observation  periods  1975-76  and  1977.  The 
proportion  of  audits  in  Classification  A  (Medical)  remained  constant 
over  time.  The  proportion  in  Classification  B  (Surgical)  decreased 
3%  in  1977  with  a  corresponding  3%  increase  in  Classification  C 
(Administrative  and  Miscellaneous  Procedures)  over  the  same  time 
period.  The  volume  of  activity  increased  approximately  2.5  times  in 
the  medical  and  surgical  classifications,  and  3.5  times  in  admini- 
strative and  miscellaneous  procedures,  indicating  a  strong  propor- 
tionate growth.  In  Table  52,  the  same  information  is  broken  down  by 
quarters.  Quarter  2-75  stands  apart  due  to  its  small  number  of  MCE 
audits  and  somewhat  different  distribution. 


The  Twenty  Most  Frequently  Audited  Topic  Categories;  A  topic 
category  is  an  aggregate,  or  class,  of  related  health  care  subjects, 
such  as  medical  diagnoses,  surgical  procedures,  or  administrative 
areas  concerned  with  the  provision  of  care.  MCE  study  titles  are 
placed  into  topic  categories  which  carry  codes  adapted  from  the 
Commission  on  Professional  and  Hospital  Activities'  Hospital  Diagno- 
sis and  Operation  groups.  Each  topic  category  is  sufficiently  broad 
to  include  several  ICDA-8  or  H-ICDA  codes.     For  example: 

Topic  Category  Topic  Category  Code  Includes 

Pneumonia  1 57  H-ICDA-2  codes 

480-486 


ICDA-8  codes 
480-486 


Caesarean  Section 


560 


H-ICDA-2  codes 
74.0,  74.2-74.9 


ICDA-8  codes 

74.6,  77.0 

and  77.2-77.9 


A  listing  of  the  topic  categories  can  be  found  in  the  MCE 
Clearinghouse  Quarterly.  /1 

Table  53  shows  the  twenty  topic  categories  most  frequently 
chosen  for  audit  in  each  observation  period.  The  rank  and  classi- 
fication are  also  given.     Some  of  the  findings  ares 


/I  Ojd^  cit. ,  July,  1978. 
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Table  52 

QUARTERLY  DISTRIBUTION  OF  MCE  AUDITS  BY  MAJOR  CLASSIFICATION 


Quarter 

Classification 

Medical 

Surqical 

Admin./ 
Misc. 

Total 

2-75 

Number  of  Audits  in 
Classification 
Percent  of  All  Audits 
Rank  of  Classification 

19 

55.88 
1 

14 

41.18 
2 

1 

2.94 
3 

34 

lOOt 

3-75 

Number  of  Audits  in 
Classification 
Percent  of  All  Audits 
Rank  of  Classification 

46 

47.92 
1 

43 

44.79 

2 

- - - - — — — ■ 

7.29 
3 

1  ' 

lOOt 

4-75 

Number  of  Audits  in 
Classification 
Percent  of  All  Audits 
Rank  of  Classification 

139 
47.77 
1 

138 
47.42 
2 

14 

4.81 

3 

291 
lOOt 

1-76 

 1 

Number  of  Audits  in 
Classification 
Percent  of  All  Audits 
Rank  of  Classification 

262 
49.25 
1 

233 
43.80 

2 

3L5 
3 

532 
1001 

2-76 

1  

Number  of  Audits  in 
Classification 
Percent  of  All  Audits 
Rank  of  Classification 

422 
48.23 
1 

404 
46.17 

2 

49           i  875 
5.60      I  100X 

3  | 

3-76 



Number  of  Audits  in 
Classification 
Percent  of  All  Audits 
Rank  of  Classification 

565 
50.27 
1 

486 
43.24 

2 

i 

73  1124 
6.49      |  lOOt 
3 

4-76 

Number  of  Audits  in 
Classification 
Percent  of  All  Audits 
Rank  of  Classification 

737 

47.365 

705 

45.308 

2 

114           1  1556 
7.327     j  1001 
3 

1-77 

Number  of  Audits  in 
Classification 
Percent  of  All  Audits 
Rank  of  Classification 

986 
47.61 
1 

!  i 

903       1      182          i  34 
43.60  i        8.79      !  1001 

t  -    j        3  ,  j 



2-77 

Number  of  Audits  in 
Classification 
Percent  of  All  Audits 
Rank  of  Classification 

1364 
48.89 

1 

 1 

1203 
43.12 
2 

223 
7.99 
3 

2790 
lOOt 

3-77 

Number  of  Audits  in 
Classification 
Percent  of  All  Audits 
Rank  of  Classification 

1224 
47.87 
1 

1084 
42.39 

2 

i  ; 

; 

249           j     2557  j 
9.74      1  100S 

3  | 

4-77 

Number  of  Audits  in 
Classification 
Percent  of  All  Audits 
Rank  of  Classification 

• 

1787 
49.37 
1 

1443 
39.86 
2 

390  3620 
10.77      j  lOOt 

1 



TOTAL 

Number  of  Audits  in 
Classification 
Percent  of  All  Audits 
Rank  of  Classification 

7555 
48.58 
1 

 1 

6659 
42.81  ' 
2 

 1  

1339           j  15553 
8.61  lOOt 

3  1 
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•  The  ranking  is  generally  consistent  from  1975-76  to  1977 
with  slight  shifts,  but  only  a  few  additions  and  dele- 
tions. The  only  topic  category  making  a  substantial 
advance  in  rank  was  "Medicinal  Agents"/  which  went  from 
ninth  in  1975-76  to  second  in  1977. 

•  Myocardial  Infarction  was  the  most  frequently  audited 
topic  category  in  both  periods  though  the  percentage  of  MI 
audits  to  all  audits  decreased  approximately  1%.  Pneu- 
monia and  Cholecystectomy  ranked  consistently  high  next  to 
Myocardial  Infarction  as  the  most  frequently  selected 
topics.  The  percentage  of  all  audits  for  these  two  topic 
data  categories  also  decreased  by  approximately  1%  from 
1975-76  to  1977,  possibly  indicating  a  trend  toward  topic 
diversification. 

•  The  top  twenty  of  19  77  account  for  fewer  audits  (41.8%) 
than  do  the  top  twenty  of  1975-76  (48%).  This  further 
supports  a  trend  toward  topic  diversification. 

The  Ten  Most  Frequently  Audited  Topic  Categories  by  Classification; 
Tables  54  through  56  show  the  ten  most  frequently  audited  topic 
categories  within  each  classification.     Some  of  the  findings  are: 

•  In  Classification  A  (Medical)  the  rank  ordering  remained 
essentially  constant  except  for  the  substantial  increase 
in  audits  related  to  medicinal  agents  in  1977.  Pulmonary 
embolism  replaced  peptic  and  duodenal  ulcers  among  the  top 
ten  medical  audit  topics  in  1977. 

•  Though  the  number  of  medical  audits  increased  2-1/2  times 
from  19  75-76  to  19  77,  the  percentage  of  the  top  ten  for 
all  medical  audits  remained  constant  at  48%. 

•  The  ten  most  frequently  audited  topic  categories  in 
Classification  B  (Surgical)  tended  to  be  the  same  in  both 
observation  periods.  Cancer  of  the  colon  replaced  herni- 
orrhaphy in  1977. 

•  Though  the  number  of  surgical  audits  doubled  in  19  77, 
the  percentage  accounted  for  by  the  top  ten  topic  cate- 
gories decreased  from  50%  to  43%  of  all  surgical  audits. 

•  There  is  some  shifting  in  the  ranking  of  the  ten  most 
frequently  audited  topic  categories  in  Classification  C 
(Administrative  and  Miscellaneous  Procedures).  Audits 
related  to  the  emergency  room  and  blood  usage  and  trans- 
fusions greatly  outnumber  the  other  topics,  and  are  half 
of  the  audits  in  1977  for  Classification  C.  Though  the 
number  of  Classification  C  topics  tripled  from  1975-76  to 
19  77,  the  percentage  of  the  top  ten  remained  constant  at 
58%  for  all  Classification  C  audits. 
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TOTAL,  1975-1977 

Percent  of 
all  in  Class- 
ification A 

iflh          o  h      ui  ui  v  r-  in 
ft  r  »  h  »  n      n  n  n 

Number  of 
Audits 

721 
586 
373 
537 
308 
286 

266 
193 
185 
135 

3590 

Percent  of 
Total  In 
Classifi- 
cation 
1977 

IAtf(D(FVN    1    fntCPIN  CD 

o  »■  h  «  » ■  e>      r»  «  ^  m  » 
ffi  h  ^  (D  ^  n      n  rj  n  n  CD 

Percent  of 
Total  In 
Classifi- 
cation 
1975-76 

10.76 
9.21 
5.33 
4.24 
3.92 
3.46 
3.19 
3.01 
2.73 
2.51 

48.36 

Number 
of  Audits 
1977 

it  v  ifl  «  n  o  i  o  n  o  in 
CD  05  m      *n  *-      o  n  m  n  o* 
^  o  «  «  n  n      in  »-  ^  •-  m 
N 

Number 
of  Audits 
1975-76 

icrsr-mioieoicomi  *- 
**>©*-eva>t^r^i0i0in  ia 
n  rx  **  o 

Rank 
1977 

(97  PSROs) 

—  n  «g>  cm  in  id  I  r»e*oeo 

Rank 
1975  76 
(73  PSROs ) 

Topic 
Category 

Myocardial  Infarction 
Pneumonia 
Diabetes  Hellltus 
Medicinal  Agents 
Gastroenteritis 
Cerebrovascular  Accident 
Peptic  and  Duodenal  Ulcers 
Congestive  Heart  Failure 
Urinary  Tract  Infection 
Hypertension 
Pulmonary  Embolism 

TOTAL 
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Conclusions 


•  Of  the  153  conditional  PSROs  (as  of  March  31,  1978),  109 
have  reported  varying  levels  of  MCE  activity. 

•  The  total  number  of  audits  reported  each  quarter  has 
increased  substantially.  Only  1,554  audits  were  reported 
in  the  fourth  quarter  of  1976;  3,690  audits  were  reported 
in  the  fourth  quarter  of  19  77. 

•  The  10  most  active  PSROs  (in  MCEs)  account  for  35%  of  all 
MCE  activity.  The  average  number  of  audits  per  PSRO  in 
the  fourth  quarter  of  1977  is  38. 

•  The  number  of  reaudits  reported  is  lagging  behind  expected 
frequencies. 

•  48.6%  of  initial  audit  topics  are  medical,  42.8%  are 
surgical,  and  8.6%  are  administrative  and  miscellaneous 
procedures . 

•  The  20  most  frequently  audited  topic  categories  account 
for  42.2%  of  all  MCE  audits;  in  the  1977  report  they 
constituted  56%  of  all  audits.  This  suggests  diversifica- 
tion in  topic  selection. 

5.3     Study;     Pilot  Variation  Rate  Analysis 
5.3.1  Objectives 

As  discussed  above,  the  variation  rate  pilot  study  is  the 
first  step  in  a  series  of  studies  which  will  assess  the  impact 
of  MCEs  on  the  quality  of  care  delivered  in  acute  care  hospitals. 
The  basic  approach  is  to  measure  changes  in  adherence  to  locally- 
set  criteria  of  good  care  between  the  time  of  audit  and  reaudit. 
The  present  study  has  both  methodological  and  substantive  objec- 
tives : 

•  To  test  and  refine  the  methodology  to  be  used  in  further 
studies. 

•  To  provide  a  "  irst  look"  at  the  kinds  of  findings  and 
inferences  which  may  result  from  such  studies. 

Substantive  findings  regarding  MCE  effectiveness  are  neces- 
sarily limited  because  they  are  based  on  available  data  from  a  pilot 
sample  of  136  MCEs  in  six  topic  categories  from  eight  participating 
PSROs.  Nevertheless,  several  preliminary  issues  can  be  addressed  by 
analyzing  these  variation  rate  data: 


203 


•  If  the  aggregate  variation  rates  show  improvement  from 
audit  to  reaudit,  such  findings  are  consistent  with 
positive  program  impact:  the  MCE  process  used  in  the 
PSROs  under  study  would  appear  to  render  care  more  uniform 
and  in  compliance  with  explicit  criteria  thought  to  be 
related  to  good  quality.  If,  however,  the  data  from  the 
present  study  show  no  improvement  or  indicate  possible 
deterioration  in  the  variation  rates,  then  the  expectation 
is  reduced  that  the  larger  studies  to  follow  will  reflect 
a  positive  impact. 

•  MCEs  can  be  expected  to  have  more  impact  on  some  criteria 
than  on  others.  The  study  will  attempt  to  distinguish  the 
kinds  of  criteria  for  which  variation  rates  rise,  fall,  or 
remain  the  same  from  audit  to  reaudit. 

As  a  pilot  of  the  variation  rate  method,  the  study  serves  the 
following  purposes: 

•  Assessing  the  degree  to  which  MCEs  in  different  hospitals 
and  PSROs  employ  common  criteria  for  the  same  topic 
category./!^  If  criteria  are  not  common,  it  will  be 
difficult  in  future  studies  to  aggregate  variation  rates 
across  study  sites. 

•  Identifying  the  commonly-used  criteria  in  the  topic 
categories  under  study  so  that  reaudit  procedures  for 
further  evaluation  studies  can  be  specified. 

•  Gathering,  whenever  possible,  evidence  regarding  the 
validity   of    routinely  recorded  MCE   compliance  data. 

The  variation  rate  findings  and  their  background  documents  were 
further  assessed  by  a  staff  physician  who  examined  the  MCE  process 
and  commented  on  the  significance  of  variation  rate  patterns  for 
assessing  quality  of  care.     Important  issues  include: 

•  Do  reductions  in  variation  rates  imply  improvements  in 
quality  of  care?  Each  topic  category  was  examined  to 
ascertain  whether  clinically  important  criteria  improved, 
or  whether  only  unimportant  ones  changed.  The  analysis 
further  sought  to  determine  whether  statistically-observed 
changes  were  apparent  or  real,  i.e.,  whether  improvements 
were  in  the  treatments  or  outcomes  themselves  or  merely  in 
the  way  they  were  recorded. 


/I  "Criteria"  are  predetermined  dimensions  of  patient  care  or 
outcome  upon  which  quality  is  assessed. 
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•  The  process  of  justification  -  i.e.,  the  retrospective 
allowance  of  exceptions  to  criteria  during  the  audit 
process  —  was  examined  for  a  subset  of  mortality  and 
major  process  criteria.  Since  the  variation  rate  data  are 
based  on  compliance  with  standards  without  considering 
possible  after-the-fact  justifications,  the  possible 
impact  of  justifications  required  consideration. 

•  The  selection  of  criteria  was  assessed,  e.g.,  whether 
common  criteria  were  also  the  most  significant  clinically, 
whether  criteria  unique  to  a  few  MCEs  (for  which  variation 
rates  were  not  computed)  were  more  or  less  significant 
than  those  common  across  studies,  and  why  common  criteria 
were  not  reaudited  in  some  MCEs. 

•  The  appropriateness  of  completed  reaudits  was  evaluated: 
whether  PSROs  or  delegated  hospitals  do  reaudits  when  they 
are  indicated,  using  the  appropriate  criteria. 

5.3.2  Data 

The  pilot  study  reported  here  is  based  on  data  available  from 
the  PSROs  which  agreed  to  participate.  There  was  no  attempt  to 
collect  primary  data  or  to  obtain  representative  samples. 

Selection  of  PSROs.  In  choosing  MCEs  to  be  used  for  this 
study,  the  evaluation  team  attempted  to  select  studies  from  those 
PSROs  which  had  the  most  experience  in  medical  care  evaluation. 
PSROs  were  thus  selected  according  to  their: 

a.  Total  number  of  MCE  initial  audits  reported  via  the  PMIS 
in  1976-77. 

b.  Total  number  of  MCEs  reported  to  have  been  completed 
with  reaudits. 

c.  Proportion  of  MCEs  which,  when  carried  out  through  the 
initial  audit  phase,  were  found  to  have  deficiencies 
requiring  the  performance  of  a  reaudit. 

d.  Duration  (in  years)  of  MCE  reporting. 

Fourteen  PSROs  were  found  to  be  in  the  top  20%  of  all  PSROs 
with  reference  to  the  above  criteria.  Of  the  14,  10  were  selected 
on  the  basis  of  interest  and  past  communications  with  staff  at 
HSQB.  Of  the  ten,  eight  were  found  to  have  immediate  access  to  MCEs 
in  the  topic  categories  utilized  in  this  study. 

Selection  of  topic  categories.  Eight  topic  categories  were 
initially  selected  by  HSQB  staff  for  use  in  the  pilot  study.  The 
reason  for  selecting  these  specific  topic  categories  lies  primarily 
in  their  frequency  of  performance  during  the  years  1975-78  through- 
out the  country  as  well  as  in  the  selected  PSROs.  As  it  turned  out, 
however,   only  six  of  the  eight  topics  could  be  used  because  there 
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were  an  inadequate  number  of  accessible  MCEs  with  reaudits  for  two 
topic  categories.  The  six  topic  categories  used  in  the  pilot  study 
include:  Myocardial  Infarction,  Pneumonia,  Diabetes  Mellitus, 
Hysterectomy,  Appendectomy,  and  Cholecystectomy.  Beyond  the  fre- 
quency with  which  they  are  chosen  for  auditing,  these  topic  catego- 
ries   exhibit    characteristics    desirable    for    program  evaluation: 

•  They  represent  patients  from  each  of  the  major  disciplines 
of  medicine  (OB/GYN,  medicine,  surgery,  pediatrics)  except 
for  psychiatry. 

•  Meaningful  criteria,  reflecting  adequate  patient  care  from 
both  process  and  outcome  standpoints,  can  be  developed  and 
are  expected  to  be  used  commonly  among  the  majority  of 
MCEs  performed  in  these  topic  categories. 

•  At  least  one  criterion  can  be  expected  to  reflect  effi- 
ciency of  care  in  MCEs  done  in  the  topic  category  (length 
of  stay  in  surgical  procedure  MCEs  and  expensive  ancillary 
services  criteria  in  medical  MCEs). 

•  Nationwide,  they  are  among  the  twenty  most  frequently 
audited    (though    not    necessarily    reaudited)    MCE  topics. 

Once  topic  categories  and  PSROs  of  interest  had  been  identi- 
fied, letters  were  sent  from  the  Director  of  HSQB  to  the  Executive 
Directors  of  the  PSROs  explaining  the  project  and  soliciting  their 
participation.  Most  were  highly  cooperative.  PSRO  staffs  were  then 
contacted  to  obtain  all  available  documentation  for  the  selected 
MCEs:  copies  of  standard  reporting  forms  (HSQB  131  and  133)  as  well 
as  MCE  process  forms  (e.g.,  PEP)  and  committee  minutes.  Both  of  the 
latter  data  sources  had  physician  and  hospital  identifiers  removed. 
A  summary  of  the  data  collection  process  is  shown  in  Table  57 . 
About  half  of  the  MCEs  came  from  one  PSRO;  the  other  seven  PSROs 
selected  for  inclusion  in  the  analysis  contributed  from  four  to 
nineteen  studies  each. 

TABLE  57 

SUMMARY  OF  VARIATION  RATE  ANALYSIS  DATA  COLLECTION 


Topic 
Category 

Number  of  PSROs 
Reporting  Studies 
Completed  through 
Reaudit 

Number  of 

Studies 

Reported 

Completed 

Through 

Reaudit 

Number  of 

Completed 

Studies 

Submitted 

for 

Evaluation 

Number  of 
Studies 
Used  In 
VR  Analysis 

Appendectomy 

6 

26 

23 

23 

Cholecystectomy 

7 

20 

20 

18 

Diabetes  Mellitus 

6 

21 

21 

21 

Hysterectomy 

8 

23 

22 

19 

Myocardial  Infarction 

8 

29 

28 

24 

Pneumonia 

5 

31 

31 

31 

TOTAL 

150 

145 

136 
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5.3.3  Methods 


The  variation  rate  (VR)  statistic,  used  in  this  study  to 
measure  deviation  of  actual  care  from  standards,  is  the  proportion 
of  patient  records  audited  on  a  particular  criterion  which  fail  to 
meet  the  standard  established  for  the  criterion. 

When  planning  an  MCE  audit  or  reaudit  on  a  given  topic,  the 
typical  medical  audit  committee  decides,  among  other  things,  the 
explicit  criteria  which  specify  important  dimensions  (or  charac- 
teristics) of  patient  care,  the  method  for  sampling  patient  records, 
and  the  sample  size  to  be  drawn.  To  each  criterion  is  assigned  a 
standard  of  compliance,  generally  a  statement  that  all  patient 
records  should  have  the  characteristic  (100%  for  desirable  charac- 
teristics) or  that  none  should  (0%  for  undesirable  characteristics). 
Allowable  exceptions  to  the  standard  for  each  criterion  are  also 
specified  in  advance.  The  sample  patient  records  are  then  examined 
according  to  the  criteria,  and  the  proportion  complying  or  not 
complying  with  the  standard  for  each  criterion  is  recorded.  These 
tallies  then  serve  as  the  basis  for  the  committee's  analysis  and 
findings. 

In  examining  the  deviations  from  standards  revealed  by  the 
tallies,  the  audit  committee  may  wish  to  exempt  certain  of  them  as 
justified,  and  take  action  only  on  the  remaining  ones.  The  process 
of  justification  must  be  carefully  distinguished  from  the  specifi- 
cation of  allowable  exceptions  to  criteria  before  the  data  are 
gathered:  the  exception  process  is  explicit;  the  justification 
process  is  implicit  and,  from  an  evaluation  standpoint,  subject  to 
unreliable  application.  For  this  reason  as  well  as  the  lack  of 
systematic  data  on  the  justification  process,  the  statistical 
analysis  of  variation  rates  in  this  study  will  be  based  on  devia- 
tions from  standards  regardless  of  any  justifications  subsequently 
invoked  when  medical  audit  committees  analyze  the  data.  The 
consequences  of  this  methodological  decision  will  be  a  subject  in 
the  clinical  discussion  (Section  5.3.6)  which  follows  the  presenta- 
tion and  analysis  of  the  variation  rate  data. 

The  variation  rates  to  be  compared,  then,  are  the  proportions 
of  patients  audited  whose  care  deviated  from  the  standard  estab- 
lished for  a  particular  criterion  before  (i.e.,  at  initial  audit) 
and  after  (at  reaudit)  corrective  actions  presumably  were  taken  in 
the  MCE  cycle  —  regardless  of  whether  the  deviations  were  regarded 
as  justified.  One  further  step  was  taken:  the  VRs  which  received 
the  most  consideration  are  those  for  criteria  which  are  common 
across  study  sites.  Criteria  were  considered  common  if  they  ap- 
peared in  no  fewer  than  five  studies  or  were  used  to  examine  in 
aggregate  100  or  more  patient  records.  A  single  variation  rate  was 
computed  for  all  patients  audited  on  each  common  criterion  within  a 
topic  category,  pooled  across  study  sites. 

Audit  and  reaudit  variation  rates  are  derived  from  different 
samples  of  patient  records  at  each  study  site.     This  is  basic  to  the 
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MCE  evaluation  process:  the  samples  at  audit  and  reaudit  represent 
successive  cross-sections  of  medical  care  before  and  after  the 
intervention.  The  key  comparison  is  between  variation  rates  at 
audit  and  reaudit  for  study  sites  (hospitals)  which  consistently 
used  a  criterion  at  both  audit  and  reaudit. 

Comparisons  of  VRs  at  audit  and  reaudit  must  take  into  account 
a  small  but  unavoidable  rounding  error  which  is  due  to  the  fact  that 
VRs  for  each  MCE  were  available  only  as  rounded  proportions.  The 
amount  of  error  which  this  contributed  to  comparisons  of  pooled  VRs 
was  estimated,  and  differences  which  did  not  exceed  these  error 
estimates  were  not  regarded  as  VR  changes. 

Classification  of  criteria.  A  quality  assurance  program  cannot 
be  expected  to  affect  all  criteria  of  care  equally.  Some  dimensions 
of  care  should  reasonably  be  expected  to  be  affected  by  changes  in 
provider  behavior,  while  others  are  less  amenable  to  control.  In 
order  to  take  this  factor  into  account,  physician-raters  were  asked 
to  classify  criteria  found  in  the  audits/reaudits  in  three  major 
categories : 

•  Process :  Those  procedural  aspects  of  diagnosis,  utilization, 
management,  and  treatment  regarded  as  directly  under  provider 
control . 

•  Preventable  Outcome:       Patient    outcomes    closely    linked  to 
provider  behavior. 

•  Non-preventable  Outcome :     Outcomes  which  are  largely  beyond 
the  direct  influence  of  medical  intervention. 

The  categorization  of  each  criterion  is  shown  in  the  tables 
presented  in  Section  5.3.5  ("Findings").  In  general,  the  mixture 
of  process  and  outcome  criteria  in  the  studies  reviewed  confirms  the 
conclusion  of  the  1977  MCE  evaluation  that  PSRO-sponsored  MCEs  were 
simultaneously  process  and  outcome  audits,  rather  than  outcome 
audits  alone. 

Hypotheses.  Once  hospital-consistent  variation  rates  were 
calculated  for  each  criterion  at  audit  and  reaudit,  a  non-parametric 
test  (Wilcoxon  signed  rank  test)/1^  was  applied  to  test  the  hypo- 
thesis that  the  variation  rates  at  reaudit  are  indeed  lower  than 
those  at  audit.  Three  such  tests  were  performed,  one  for  each  of 
the  three  classes  of  criteria  defined  above. 

Validity  substudy.  A  competing  hypothesis  to  explain  observed 
changes  in  VR  is  that  audit  committees  may  have  altered  their 
criteria  or  methods  of  data  collection  between  audit  and  reaudit. 
For  this  pilot  study,  a  subset  of  criteria  was  examined  to  ascertain 
the  degree  to  which  definitions  and  instructions  remained  constant. 


/I  Conover,    W.J.,    Practical  Non-Parametric  Statistics    (N.Y. :  John 
Wiley  &  Sons,   Inc.,   1971)  pp.  206-215. 
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Clinical  assessment.  The  results  of  the  variation  rate  study 
together  with  supporting  documents  were  then  subjected  to  clinical 
analysis.  In  addition  to  the  statistical  findings,  a  staff  physi- 
cian examined  criteria  which  were  not  sufficiently  common  to  cal- 
culate variation  rates.  Clinical  assessment  went  beyond  the  sta- 
tistics gathered  and  was  used  to  evaluate:  (1)  the  importance  of 
changes  observed  (or  of  the  absence  of  changes);  (2)  the  validity  of 
using  variation  rates  based  only  on  common  criteria  to  assess 
quality  of  care;  and  (3)  the  appropriateness  of  reaudits.  While  the 
methodology  of  clinical  assessment  is  largely  implicit  judgment, 
this  judgment  is  based  upon  training  in  accepted  methods  of  practice 
and  in  medical  auditing.  Analyses  were  guided  by  the  objectives 
described  in  Section  5.3.1  above. 

5.3.4  Limitations 

The  sample  of  MCEs  and  audited  records  used  in  this  pilot  study 
is  too  small  and  unrepresentative  to  permit  the  drawing  of  infer- 
ences on  a  nationwide  basis.  This  limitation  applies  to  inferences 
about  both  patients  in  the  chosen  topic  categories  and  the  over- 
all success  of  MCEs  nationwide. /1  Even  among  the  PSROs  studied,  the 
sample  MCEs  are  unevenly  distributed.  The  PSROs  which  participated 
in  the  pilot  study  have  demonstrated  the  technical  capability  and 
leadership  required  to  encourage  early  compliance  with  the  PSRO  pro- 
gram* s  requirement  that  reaudits  be  performed;  to  this  extent  they 
are  exceptional.  MCEs  in  PSROs  or  topic  categories  not  examined 
here  may  be  excellent  and  show  large  improvements  in  variation 
rates,  or  they  may  not.  The  findings  regarding  variation  rates 
among  the  pilot  sample  may,  however,  give  some  indication  of 
what  can  be  expected  in  future  studies. 

Second,  the  data  for  the  present  study  are  limited  in  the  sense 
that  only  the  information  routinely  recorded  was  available  for 
analysis.  Variability  in  chart  review,  abstracting,  and  recording 
may  contribute  unknown  errors.  Some  data  were  simply  not  obtainable 
in  the  time  available  for  the  study.  Yet  the  primary  data  for 
variation  rate  analysis,  the  criteria,  standards,  and  tally  per- 
centages, are  considered  to  be  among  the  most  reliably  recorded  and 
readily  available  pieces  of  information  on  the  MCE  process.  This 
limitation  will  be  ameliorated  in  the  1979  study,  in  which  available 
data  will  be  supplemented  by  considerable  primary  data  collection, 
including  independent  reauditing  on  uniform  criteria. 


/1  Inference  to  both  audited  and  unaudited  patient  populations  in 
the  selected  topic  categories  would  require  the  assumption  that 
patient  records  sampled  for  inclusion  in  MCEs  adequately  repre- 
sent all  patients  with  a  given  diagnosis  or  procedure.  While 
this  issue  will  be  evaluated  in  the  course  of  the  research,  the 
assumption  is  not  now  warranted. 
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A  third  limitation  of  variation  rate  analysis  is  that  improve- 
ments in  VRs  are  necessarily  small  on  criteria  where  care  is  good  to 
begin  with.  Many  of  these  criteria  are  not  reaudited;  where  they 
are,  they  may  show  little  or  no  improvement  simply  because  there  is 
little  to  be  achieved.  This  problem  can  be  controlled  by  careful 
attention  to  the  baseline  variation  rates  at  the  initial  audit,  and 
by  clinical  judgment  about  the  importance  of  change  or  lack  of 
change  in  VRs.  The  attempt  to  go  beyond  reported  VRs  through 
clinical  assessment  is  one  of  the  innovations  of  the  present  study. 

Fourth,  each  MCE  is  a  study  of  the  care  rendered  to  all  pa- 
tients in  the  specified  topic  area.  At  no  point  in  the  MCE  process 
or  in  this  evaluation  are  Federal  patients  distinguished  from 
non-Federal  patients.  The  PSRO  program  assumes  that  improvements  in 
quality  of  care  will  affect  all  patients  alike;  this  assumption 
cannot  be  tested  with  the  available  data. 

Fifth,  a  limitation  intrinsic  to  the  study  design  is  the 
inability  to  measure  global  effects  of  the  program.  If  the  MCE 
program  improved  quality  of  care  simply  by  making  physicians  and 
others  aware  that  their  performance  would  be  scrutinized,  then  the 
program  effect  would  precede  the  initial  audit.  This  impact  would 
not  be  detected  through  variation  rate  analysis  unless  appropriate 
controls  were  included  in  a  quasi-experimental  design.  There  is 
reason  to  believe,  on  the  other  hand,  that  such  global  effects  are 
not  large  or  sustained,  and  that  the  program  would  have  to  show 
concurrent  improvements  in  quality  in  order  to  be  considered  effec- 
tive . 

Finally,  this  pilot  study  employs  no  control  group.  Thus  it 
cannot  be  asserted  with  confidence  that  changes  observed  are  due  to 
the  influence  of  the  PSRO  program;  other  plausible  hypotheses  cannot 
be  ruled  out. 

5.3.5  Findings 

The  tables  which  follow  (Tables  58  through  63)  present  the 
variation  rates  found  for  each  common  criterion  in  the  six  topic 
categories  studied,  along  with  the  numbers  of  audits,  reaudits,  and 
patients  used  to  calculate  each.  The  first  column  of  each  table 
lists  the  criteria  examined.  Those  in  the  bottom  portion  were  not 
sufficiently  common  to  meet  the  criteria  for  calculating  VR  changes 
(5  or  more  studies  or  100  or  more  patient  records  at  both  audit  and 
reaudit)  but  were  used  frequently  enough  to  yield  initial  audit 
VRs.  They  do  not  figure  further  in  the  statistical  analysis,  but 
are  presented  to  show  the  types  of  criteria  which  were  not  fre- 
quently reaudited  and  their  variation  rates  at  the  initial  audit. 
Some  of  these  criteria  will  be  examined  in  the  clinical  discussion 
(Section  5.3.6). 


210 


Chanqe 
Between 
Initial  and 
Reaudit 

Down  . 1 1 
Up  .04 

Down  . 36 

Rounding 
Error 

n  n  m 
o  o  o 

Variation  Rates 

Reaudit 

on  © 

Initial 
Followed 
By  Reaudit 

cm 

n  n  *7 

Initial 
Audit 

oo  o  m 
N  n  r> 

VD             0^  ^ 
O               O  O 

Initial  Audits 
Followed  by  Reaudits 

No.  of  Patients 
(Reaudit) 

W  5  ri 
n  cm 

No.  of  Patients 
(Initial  Audit) 

«-  O  CD 
«-  «-  T 

Number 
Performed 

m  r  «? 

Initial  Audits 
(Total) 

No.  of 
Patients 

(M  CO 

t-               01  CO 

*7          r-  o 
r-  r» 

Number 
Performed 

in  r» 

n  o 

Criteria 

Common  Criteria 

PROCESS: 

Length  of  Stay 

Positive  Pathology  Report 

PREVENTABLE  OUTCOME: 
Appropriate  Wound  Healing 

HUW    rKtV  EininDLfi    UU  IVV/ITEi 

Others 

Physical  Exam  Done 

Gangrenous  or  Perforated 
Appendix 

Wound  Infection 

211 


i  E 

1 


Change 
Between 
Initial  and 
Reaudlt 

Down  . 13 
Down  .03 

Down  .08 

Down  .05 
Down  .09 
Down  .08 

0 

0 
0 
0 
0 

Down  .04 
Down  .04 
Down  .03 

0 
0 

Rounding 
Error 

m  »-     *~         n  n  n            cn     *n  ^  ^  m     tn  cm         *~            *~  »- 
ee     e        eee            e     eeeo     ooe     ©            ©  © 

Variation  Rates 

41 

•0 
9 

I 

Mr*     "~         co  *p  it?                   ooeo     in     •*                   ©  ^ 
ee     o         oof            e     eeee     eee     e  ee 
..      .          ...             •      ....       •  •  •      •  .. 

Initial 
Followed 
By  Reaudlt 

in  in     ft                             •*     eeee     p-  «•>  in  © 
*"  °     °        *"     "            e     eeee     ooe     ©  ee 

Initial 
Audit 

cm*     in         ki  a  ■                   e  b  ^  o     t  n  n     p-  e*» 
»^                o  <-  *-            e     oeoo     ooe     *-  oo 

..            e                    ...                          .            ....            ...            .  .. 

e 
• 

u 

1 

z 

of  Patients 
( Reaudlt ) 

©ft     in         cm     «»            oi     *  »=  «  ft     nma  em 
n  d                o  o  is            o*     ^  o  ft  o     in^**     ft            od  r- 

n        V              IN  «-  1-1                   ^        w        1-              IN  >-  IN        CM                   ^  CM 

Initial  Audits 
Followed  by  Reaudlta 

No.  of  Patients 
(Initial  Audit) 

O  ©        W             OD  0t  ©                  lft       49  f**        F*        ©        fs*i       |^                 a>>  \g 

n  ft      CD         »«n             im      in  o  ft  '      Of  r-  eo      «-             in  ft 

Mf        w            IN  «-  «                  CM       V  CU  «•  <N        N  «-  CM       «*>  CM 

Number 
Performed 

in  »-     •»         h  bi  «            r>     in  in  in  14)     ft  in  io 

«-                      ~                                                  p>            in  r» 

Initial  Audits 
(Total) 

No.  of 
1  Patients 

eon     ^         e  cm  cm            t»     ft  p-  •»  o     r»«-fi     <£>  coo 
p.  ft     in         in*-m            cm     to  p.  p.  a)     in  in  mp     to            n  it 
cm  to     cm         m  cm  to            Mf     cm  cm  n  cm     MToim     m            cm  in 

o 

p> 

Number 
Performed 

00  \C      m>         ft  to  to             o      o>p»op»      mom      <n             m?  in 

m 

• 

Criteria 

Common  Criteria 

PROCESS  i 

Surgery  Justified  By 
Diagnosis 

Abnormal  Pathology  Report 

Appropriate  Use  of  Cholan- 
giogram  for  Diagnosis 

Xray  Diagnosis  of  Non- 
Functioning  Gall  bladder 
Appropriately  Hade 

Appropriate  Lab  Tests 

Length  of  Stay 

PREVENTABLE  OUTCOME: 

Hound  Infection 

Deep  Venous  Thrombosis  or 

Pulmonary  Embolus 
Post  Operative  Fever 
Peritonitis 
Hound  Dehiscence 
Pneumonia  and/or 

Atelectasis 
Ambulatory 

Appropriate  Mound  Healing 
Afebrile  in  Appropriate 
Period  of  Time 

HON— PREVENTABLE  OUTCOME: 

Hemorrhage 
Mortality 

ii 
3     a  u 

212 


8 


Change 
Between 
Initial  and 
Reaudlt 

Down  . 13 

Down  .17 
Down  .13 
0 

Down  .04 
0 

Rounding 
Error 

n         «     nnn                 rt  «n 
o         o      o  ©  o                 o  e 

Reaudlt 

cm         cm      •»  io  e»                 c>  n 
©         e      e  n  m                 o  © 

Variation  Rate. 

Initial 
Followed 
By  Reaudlt 

cm         in         0>  e>                 n  cm 
o           •-      CM  •»  CM                   »»  e 

Initial 
Audit 

r»           *=*       N  D  o                   o  cm 

©              »-        »-  •»  CM  O 

v>       IB       CM  CO 

«-      ©     e  © 

\ 

Initial  Audita 
Followed  by  Reaudlta 

No.  of  Patients 
(Reaudlt) 

2                h«n                 2  2 

No.  of  Patiente 
(Initial  Audit) 

co         in     •»  co  co                co  r» 
e         cm     f-  id  cm  o 
cm         <•     n  m  n                m  cm 

Number 
Performed 

CD           in      o  ♦»  0>                   B>  CO 

Initial  Audita 
(Total) 

No.  of 
Patienta 

•»         CO     o  id  M                 m  m 

r>         r*»      ib     m                 io  in 
r»         •»     in  io  f»                in  •» 

m     io  ■  •*■  n 
•»     io     m  •- 

CM        j0        •»  10 

£ 

*-         m     m  in  *■                u>  mp 
cm                   »-  c*  »- 

u 

8.6. 

in     co  '  **■  m 

Criteria 

Common  Criteria 

PROCESS > 

Abnormal  Glucose  Tolerance 
Test  or  Elevated  Blood  Su 

Diabetes  Uncontrolled  or  SI 
of  Serioua  Infection  as 
Indication  for  Admission 

Repeat  Blood  Sugar  and 
Urinalyala 

Diaeaae  Counseling 

Length  of  Stay 

Laboratory  Proof  of  Control 
of  Blood  Sugar 

NON-PREVENTABLE  OUTCOME 

Mortality 

Others 

Appropriate  Uaa  of  Antl- 

Hypergly cemi  cs 
Hypoglycemia  Developing  in 

House 

Hyperglycemia  Persists 

Despite  Therapy 
Infection  Developing  After 

Admission 

213 


U  M 


Change 
Between 
Initial  and 
Reaudlt 

Down  .21 
Up  .03 
Down  .  06 

0 

Down  .04 

Rounding 
Error 

«  n  >•             »-  m 
o  o  ©              o  s 

Reaudlt 

.05 
.35 
.05 

.000 
.03 

ariation  Ratei 

Followed 
By  Reaudlt 

is  «  •-            o  r- 
n  n                 o  o 

> 

Initial 
Audit 

«»  m  »-             o  in 
©  © 

»-  CM  ©  ©OOOOO 

Initial  Audits 

No.  of  Patients 
( Reaudlt ) 

ninn             oo  •» 

©  Is*  ^                  N  01 
CM  «- 

No.  of  Patients 
(Initial  Audit) 

©      in             ©  *n 
S  $  2             !-  n 

Number 
Performed 

•»  ©  v             n  « 

Initial  Audits 
(Total) 

No.  of 
Patients 

•»  r*  co             in  cd 
n  ui              n  \o 

•-cMcftirtn.-^inflD 
^  in  •»       -  m  in  "a1  en 

Number 
Performed 

o  n  n             v  n 

Criteria 

Common  Criteria 

PROCESS: 

Appropriate  Admission 
Pap  Test  in  Last  6  Months 
Length  of  Stay 

PREVENTABLE  OUTCOME: 

Ambulatory 
Fever 

NON-PREVENTABLE  OUTCOME: 

Others 

EKG  If  Age  >  40 

Positive  Pathology  Report 

Post  Op  Infection 

UTI 

ILEUS 

Hound  Dehiscence 
Preventable  Injury 
Mortality 
Hemorrhage 

214 


u  & 


c  c  c  c 


11 


■H  b 

•a  e 
c  b 


8  5 

■H  -H  K 

~  0 


4J  id 
•H  9 
C  < 


io  ^  r»  o  « 
a  ifl  «  n  n  n 

fN  T-  (N  (N 


■rt  B 
■0  O 

g  e 


e  -h 
e  >c 
■H  g 


■ 

O  U 


O  M  Ch  CO 


in  in 
in  CN 


ic  co  en  m  q)  o 


"  P 
■h  H 


51  o  oi  m  » 
it  t  o  a  ^  f 
in  id  io  «s  n  m 


*  ^  in  oi  ©  m 


CO  CO 

vO  1*1 


r-  03 

M  IN 


'  1     !  o 


D  <  *> 

a  e 

m  >i  h  s  H 

>i  r  ->  o  tr 

6        ■  «-  to  C  M 

4J  lb   4J  -H  — '  0 

(fl    0    B  b  ■  b 

•H  O  O  0  0" 

M   e   b  B  ±>  C 

o  5  a  a  &>  -h  -h 

-i  c  v  a  c  u 

j=   (J   b  -i  -i  0  c 

jj  -i  a  ■o  o  c 

a>  a  a  e 

C        <  M  15  U  _ 

J  W  W  S  £ 


■H   J  -H 

>  o 


O  X 


r  n  N  1*1;  n  »•  ^  m  n  r-  (ij 


<9^1£C*CD*£^CNOOf"ll0 


b  e  c  s 

■  4J  «H  9  0 

I  «  £  S>  £  S 

5,  >  4J  b   «  S 

N  »  -H  «  b 

C  S  .C  O  b 


«5 


Q  4J   e   9  B 


■oo  us 

b  4J  a  o  o. 

B  *  O 

v  B  cs 


J=  -rt   O  <  6  b 

0   b  4J  a  B 

U  £   B  -h  U 

B  -I  "H  B  b  >i 

f  fc.   9  -H  9 

A  b  O  B 


215 


Change 

Initial  and 
Reaudlt 

Down  .23 

Down  .21 

Down  .44 
Down  .29 

Down  .29 
Down  .16 
Down  .19 
Down  .09 

Down  .06  J 

Rounding 
Error 

*-      cm      r>  n         n  n  n  r  r> 
o      ©      e ©          ©  ©  ©  ©  o 

Reaudlt 

o      •*      n  <-         i-  o  n  r-  o 

ition  Rates 

Initial 

Followed 
By  Reaudlt 

*-     m     *r  n         n  n  «  * 

m     tc     to            «»  n  •»  n  oo 

•         •••              ....  o 

Varl, 

1 

Initial 
Audit 

in     •#     •»  «o         9  ^  ©  ui  <c 
nioiocn         m  «-  *  n  o 

n     o>      ic  ^  v     »o  *"  h  in 
m      «•      o  ©  ©  ©  ©  ©  ©  •» 

Initial  Audits 
Followed  bv  Reaudita 

No.  of  Patients 
(Reaudlt) 

^      p.         sd         i/i  r*-  m  i*i  n 
r>      cm      io  r»         '  n  o  n  •» 
«"      *•      pi  «-         n  n  pi  »- 

No.  of  Patients 
(Initial  Audit) 

co      o      r>  »-  nr-oin 
©                io         r-  w  •*  « 
in      n  [n         ^  ^  m  n  CD 

CM 

Number 
Performed 

(NiootCD         •*  m  o  to  oo 

Initial  Audits 
(Total 1 

No.  of 
Patients 

m      iri      n  n         ^      r-~  m 
■  m     n         in            \o  n  r  o 
^•fNini")         go  r*  v  in  r» 

M       IP       *       Mn  o>  f  1  f 

*     «-     i»  to  •»•»©»-  to  r* 
*y      oj      ^  «  n  n      ci  »• 

Number 
Performed 

n      r-      (n  »-         *t  m  <t  *- 

«»  «-             M  M  *-  M 

r>      to      in  «  «  c  •)  <f  m  in 

Criteria 

Common  Criteria 

PROCESS : 
CBC 

Smear  or  Bronchial 

Secretions 
Use  of  Antibiotic  with 

Bacterial  Investigation 
Outpatient  Followup  Arrange 
Documented  Improvement  By  | 

Physieal  Exam  and/or  chei 

Xray 

Chest  X-ray  Validation 
Sputum  Culture  and 
Sensitivity 

PREVENTABLE  OUTCOHE: 

NON-PREVENTABLE  OUTCOHE  I 

Mortality 

Others 

Respiratory  Distress  Used 

as  Admission  Criterion 
Examination  of  Chest 

Documented 
Chest  X-ray  Completed  and 

Recorded 
Drug  Reaction 
Atelectasis 
Prolonged  Fever 
Unresolved  Pneumonia 
Empyema  or  Abscess 
Pleural  Effusion 
Document  Discharge 

Instructions 

216 


In  each  of  Tables  58  through  63,  the  column  headed  "Initial 
Audits  (Total)"  shows  the  total  number  of  MCE  studies  involving  each 
of  the  indicated  criteria,  whether  or  not  followed  by  reaudit;  the 
number  of  patients  involved  is  also  shown.  The  next  three  columns 
show  the  number  of  initial  audits  which  were  followed  by  reaudit  and 
the  number  of  patients  involved  at  each  of  these  two  stages. 
Variation  rates  are  then  shown  in  three  columns;  the  first  column 
covers  all  initial  audits,  the  second  covers  only  those  followed  by 
reaudit,  and  the  third  covers  reaudits  alone.  The  final  two  columns 
list,  respectively,  the  rounding  error  associated  with  that  particu- 
lar criterion  (see  earlier  discussion)  followed  by  the  observed 
change  for  those  studies  where  initial  audits  were  followed  by 
reaudit.  The  latter  measure,  as  indicated  earlier,  was  disregarded 
unless  it  exceeded  the  rounding  error. 

Overall  findings.  Table  64  presents  a  summary  of  the  results 
for  all  common  criteria.  Results  were  classified  "no  change"  if  the 
observed  difference  in  VR  between  initial  audit  and  reaudit  fell 
within  the  stated  rounding  error;  otherwise  they  were  classified  as 
"decreased  VR"  or  "increased  VR"  as  the  case  may  be.  Because  of  the 
large  number  of  cases  involving  decreased  VRs,  the  results  are 
presented  in  increments  of  .05.  Since  decreases  represent  possible 
successes  of  the  MCE  process,  MCEs  clearly  were  more  successful  in 
reducing  variations  in  process  criteria  than  in  the  other  two 
classifications . 

TABLE  64 

CHANGE  IN  VARIATION  RATE  BETWEEN  AUDIT  AND  REAUDIT 
BY  TYPE  OF  CRITERION 


Number 

of  Studies 

with  Results  Shown  Below 

Type  of 
Criterion 

Increased 
VR 

No 
Change 

Decreased 

VR 

.01- 
.05 

.00+ 
Error* 

.01- 
.05 

.06- 
.10 

.11- 
.15 

.16- 
.20 

.21- 
.25 

.26- 
.30 

.31- 
.35 

.36- 
.40 

.41- 
.45 

Process 

2 

4 

3 

7 

5 

4 

3 

2 

0 

0 

1 

Preventable 
Outcome 

0 

7 

6 

1 

0 

0 

0 

0 

0 

1 

0 

Non-Pr even tat 
Outcome 

le 

0 

4 

2 

0 

0 

0 

0 

0 

0 

0 

0 

TOTAL 

2 

15 

11 

8 

5 

4 

3 

2 

0 

1 

1 

♦Included  as  "no  change"  are  those  increases  or  decreases  which  do  not  exceed  an  estimated  rounding 
error. 
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Process  criteria*  Thirty-one  process  criteria  were  sufficient- 
ly common  among  the  MCEs  for  calculation  of  aggregate  VR  changes. 
The  following  table  shows  the  results  for  process  criteria  among  the 
six  topic  categories: 


TABLE  65 

VR  CHANGE  SUMMARY:     PROCESS  CRITERIA 


Topic  Category  Decreases        No  Change  Increases 

Appendectomy  1  -  1 

Cholecystectomy  6  -  - 

Diabetes  Mellitus  3  2  - 

Hysterectomy  2  1 

Myocardial  Infarction  5  2  - 

Pneumonia  8  -  - 


To  test  the  hypothesis  that  there  was  improvement  (reduction) 
in  VRs  among  process  criteria  beween  audit  and  reaudit,  the  Wilcoxon 
signed  rank  test  was  applied.  This  non-parametric  test  takes  into 
account  not  only  the  direction  of  each  VR  change  but  also  its 
ordinal  magnitude.  Applying  the  test,  the  aggregate  decrease  in  VRs 
was  found  to  be  significantly  different  from  zero  (p  <  .05). 

Preventable  outcome  criteria.  Variation  rate  changes  for 
those  outsomes  judged  to  be  relatively  responsive  to  the  quality  of 
medical  care  (N  =  15)  showed  the  following  pattern: 


TABLE  66 

VR  CHANGE  SUMMARY:     PREVENTABLE  OUTCOME  CRITERIA 


Topic  Category  Decreases        No  Change  Increases 

Appendectomy  1  -  - 

Cholecystectomy  3  6  - 

Diabetes  Mellitus  1  -  - 

Hysterectomy  1  1  - 

Myocardial  Infarction  2  -  - 

Pneumonia  -  -  - 
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As  in  the  case  of  process  criteria,  the  Wilcoxon  signed  rank 
test  showed  the  decrease  in  VRs  for  preventable  outcome  criteria  to 
be  significantly  different  from  zero  (p  <  .05). 

Non-preventable  outcome  criteria.  The  six  criteria  which 
measured  outcomes  judged  to  be  relatively  unresponsive  to  the 
quality  of  medical  care  showed  the  following  pattern: 


TABLE  67 

VR  CHANGE  SUMMARY:     NON-PREVENTABLE  OUTCOME  CRITERIA 


Topic  Category  Decreases        No  Change  Increases 

Appe  nde  c  t omy 
Cholecystectomy 
Diabetes  Mellitus 
Hysterectomy 
Myocardial  Infarction 
Pneumonia 


The  sample  size  for  this  class  of  criteria  is  too  small  to 
apply  the  Wilcoxon  test,  considering  that  half  the  criteria  are 
exact  ties  (change  in  VR  =  .00)  and  cannot  be  used  in  the  test. 
Inspection  of  the  table  shows  that  little,  if  any,  change  has  taken 
place. 

Validity  study.  Perhaps  the  most  serious  challenge  to  the 
validity  of  the  VR  calculations  is  the  possibility  that  medical 
audit  committees  changed  the  wording  of  criteria,  exceptions,  or 
data  collection  instructions  from  audit  to  reaudit,  thus  producing 
spurious  reductions  in  variation  rates.  The  usefulness  of  MCE  data 
for  research  purposes  was  checked  by  asking  reviewers  to  examine  the 
criteria,  exceptions,  and  data  collection  instructions  used  at  audit 
and  reaudit  for  subset  of  MCEs  in  two  topic  categories  (pneumonia 
and  cholecystectomy)  to  see  whether  changes  in  these  factors  could 
account  for  changes  in  VRs.  By  and  large,  they  could  not:  in  all 
but  ten  of  79  comparisons,  the  reviewers  found  that  none  of  these 
factors  had  changed  from  audit  to  reaudit.  Where  changes  were 
found,  the  reviewers  judged  that  the  change  would  decrease  the 
reaudit  VR  in  three  instances,  not  affect  it  in  four  instances,  and 
create  uncertain  changes  in  three.  This  cursory  examination  of 
audit/reaudit  reliability  thus  tends  to  reduce  the  suspicion  that 
changes  in  observed  VRs  can  be  accounted  for  by  systematic  bias  due 
to  loosening  of  criteria  between  audit  and  reaudit.  When  reviewers 
were  further  asked  to  judge  the  validity  of  VR  changes  in  the  same 
subset  of  criteria,  however,  they  supported  the  validity  of  the 
findings  only  63  percent  of  the  time.  The  main  doubts  about  the 
validity  of  the  data  were  due  to  (a)  changes  in  criteria,  excep- 
tions,   or   instructions   as   noted   in  the   reliability   study   and  (b) 
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instances  in  which  criteria  which  required  judgement  in  their 
application  lacked  clearcut  instructions  for  data  gathering. 

Conclusions*  The  data  show  a  definite  trend  toward  reduction 
of  variation  rates  for  process  criteria,  evident  in  both  the  table 
of  changes  and  the  findings  of  the  Wilcoxon  test.  For  preventable 
outcomes,  the  tables  show  a  less  obvious  trend,  with  as  many  cri- 
teria showing  no  change  as  improvements;  however,  the  magnitude  of 
some  of  the  improvements  was  enough  to  cause  the  Wilcoxon  test  to 
show  a  significant  aggregate  .reduction  in  VRs.  For  non-preventable 
outcomes,  where  the  expectation  of  improvement  is  less,  no  apparent 
trend  was  detected. 

In  general,  the  pattern  of  findings  is  consistent  with  the 
expectation  that  criteria  which  are  most  closely  related  to  the 
behavior  of  physicians  and  other  health  care  practitioners  will  be 
most  affected  by  MCEs.  This  pattern  tends  to  support  the  validity 
of  the  variation  rate  measure.  The  pattern  may,  however,  seem  at 
first  glance  to  be  artif actual:  process  VRs  are  likely  to  be  higher 
than  outcome  VRs  at  initial  audit,  leaving  more  room  for  improvement 
at  the  reaudit.  Among  the  topics  studied,  it  would  indeed  be 
surprising  to  find  a  case  fatality  rate  (i.e.,  variation  from  a 
standard  of  zero  mortality)  as  high  as  some  of  the  initial  process 
variation  rates,  the  overall  mean  of  those  rates  being  25.4%.  Over 
all  the  common  criteria,  the  correlation  between  the  amount  of 
variation  found  at  the  initial  audit  and  the  amount  of  change  from 
audit  to  reaudit  is  high  (r  =  .80).  One  way  of  dealing  with  this 
problem  is  to  compute  the  ratio  of  the  observed  change  relative  to 
the  amount  of  change  possible  —  i.e.,  the  proportion  by  which  the 
initial  variation  rate  is  reduced  rather  than  the  absolute  differ- 
ence. This  could  not  be  done,  however,  in  the  present  study  due  to 
the  small  samples  involved  but  should  be  considered  for  future 
studies . 

5.3.6    Clinical  Discussion 

This  subsection  provides  an  analysis  of  four  questions: 

1 .  Do   changes   in  variation  rate  imply  changes  in  the  actual 
care  received  by  patients  or  in  patient  outcome? 

2.  What   is   the   effect  of   justification  of  variations  on  the 
overall  influence  of  an  MCE? 

3.  What  are  the  implications  of  criteria  selection;  e.g. 
are  commonly  chosen  criteria  also  critical  clinical  crJ 
teria? 

4.  Are  topics  for  reaudit  appropriately  chosen? 
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Considerable  caution  is  required  in  the  attempt  to  place  VR 
findings  in  a  clinical  perspective.  The  inevitable  errors  associ- 
ated with  abstracting  clinical  records  must  be  considered.  While 
care  was  taken  in  the  VR  analysis  to  ensure  that  these  errors  were 
reduced  to  a  minimum,  uncertainty  remains  concerning  the  meaning  of 
observed  variation  rate  changes.  It  is  unclear  in  some  instances 
whether  the  observed  changes  reflect  documentation,  or  actual 
practice.  There  is  another  difficulty  in  the  fact  that  many  hos- 
pitals use  complex,  multiple  choice  criteria;  in  such  cases,  if  one 
of  a  group  of  characteristics  were  present,  the  criterion  was  deemed 
satisfied,  and  the  record  might  fail  to  indicate  which,  if  any,  of 
the  other  characteristics  were  not  present.  While  this  may  be  a 
satisfactory  approach  for  the  practitioner,  it  hampers  research 
efforts    which    seek    to   establish   cause    and   effect  relationships. 

Analysis  of  Question  1:       Do  changes  in  variation  rate  imply 
changes  in  the  actual  care  received  by  patients  or  in  patient 
outcome? 

This  is  probably  the  most  critical  question.  The  strongest 
endorsement  of  the  MCE  program  would  be  a  marked  improvement  in 
patient  care  and  outcome  noted  at  reaudit.  While  it  was  found  that 
outcome  VRs  fluctuated  little  between  audit  and  reaudit,  process 
criteria  apparently  were  influenced  by  the  MCE  program  in  that 
compliance  with  the  majority  of  process  criteria  was  found  to  be 
improved  at  the  time  of  reaudit. 

Process  Criteria.  The  largest  improvement  in  process  criteria 
occurred  with  bacterial  pneumonia,  in  which  compliance  rose  in  many 
instances  by  twenty  to  thirty  percent.  In  spite  of  this,  the 
clinical  implications  of  improved  adherence  to  process  criteria  in 
the  case  of  bacterial  pneumonia  remain  somewhat  equivocal.  First, 
considering  criteria  related  to  X-rays,  although  there  was  an 
apparent  change  in  performance,  leading  to  an  increased  number  of 
chest  X-rays  being  ordered  and  reported,  this  change  failed  to 
produce  zero  variation  rates,  i.e.,  there  remained  a  number  of 
exceptions  noted  at  reaudit.  Secondly,  some  of  the  X-ray  criteria 
were  part  of  multiple  choice  criteria,  rendering  it  difficult  to 
determine  what  really  took  place.  Nevertheless  the  X-ray  VR  reduc- 
tion helps  to  confirm  the  major  hypothesis  of  this  study,  i.e.,  that 
by  insisting  on  compliance  with  explicit  criteria,  at  least  for 
process,  MCE  can  induce  changes. 

Clinically,  very  few  patients  with  bacterial  pneumonia  have 
a  negative  chest  X-ray.  There  are  a  small  number  of  patients  who 
have  chest  X-rays  taken  outside  the  hospital  and  are  then  admitted, 
generally  with  a  note  to  that  effect  by  the  physician.  Without 
access  to  the  original  charts,  however,  the  question  of  what  hap- 
pened to  the  remaining  cases  cannot  be  resolved. 
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While  changes  in  X-ray  related  criteria  are  difficult  to 
identify  and  document,  other  aspects  of  bacterial  pneumonia  audits 
are  more  straightforward.  For  example,  the  use  of  antibiotics  with 
a  proper  bacteriological  investigation  declined  from  a  variation 
rate  of  64%  to  20%.  This  difference  of  44%  represents  a  very  real 
improvement  in  care;  it  is  more  than  a  matter  of  documentation.  If 
no  test  had  been  ordered,  there  would  not  be  a  laboratory  report 
in  the  chart.  If  the  test  had  been  done,  but  the  physician  had 
neglected  to  note  the  fact  in  the  chart,  the  laboratory  report  would 
still  stand  as  evidence  for  performance. 

Here  again,  most  physicians  would  agree  that  the  best  treatment 
is  to  use  an  antibiotic  which  has  the  greatest  influence  on  the 
pathogenic  organism.  This  can  only  be  done  if  the  organism  is 
identified. 

These  two  criteria  —  chest  X-rays  and  the  use  of  antibiotics 
—  represent  ways  in  which  MCE  can  influence  actual  care.  Similar 
changes  in  process  were  found  in  the  study  of  diabetes  mellitus. 
Beyond  mere  process  changes,  however,  compliance  with  criteria 
requiring  repeated  blood  sugar  studies  is  sound  medical  care,  as 
the  goal  of  such  tests  is  to  improve  the  control  of  the  disease. 
In  myocardial  infarction,  to  cite  another  example,  the  greater 
admission  of  patients  to  CCU  should  have  an  effect  on  the  quality 
of  the  care  given. 

Preventable  Outcomes.  These  are  chiefly  complications  which  in 
theory  should  not  occur  if  the  physician  is  vigilant  and  the  care  is 
good.  An  example  of  this  is  a  wound  infection  after  "clean"  elec- 
tive surgery. 

Less  striking  results  were  noted  in  this  category.  Compliance 
changed  by  barely  noticeable  margins.  The  implications  for  relating 
these  small  changes  to  actual  change  in  practice  are  highly  uncer- 
tain. One  of  the  larger  changes  noted  was  a  10%  reduction  in 
the  occurrence  of  congestive  cardiac  failure  in  myocardial  infarc- 
tion. This  is  suggestive  of  improved  practice,  although  the  infor- 
mation from  the  MCEs  themselves  is  insufficient  to  identify  actual 
relationships.  We  can  only  speculate  about  the  adoption  of  more 
precise  indications  for  the  use  of  cardiotonics,  and  possible 
greater   care   in  the  amount  of  intravenous  fluid  administered. 

Sometimes,  no  change  in  variation  rate  was  registered.  Allow- 
ing for  the  problem  of  abstracting  reliability  and  its  associated 
pitfalls,  the  relative  magnitude  of  the  residual  variation  at  audit 
and  reaudit  deserves  comment.  In  the  cholecystectomy  series,  for 
example,  all  of  the  preventable  outcome  criteria  were  reaudited.  In 
this  group  of  nine  criteria,  four  changed  by  2-4%  and  five  did  not 
change  at  all. 
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Improved  compliance  with  preventable  outcome  criteria  was  noted 
in  some  of  these  topics.  This  was  particularly  noteworthy  in  the 
appendectomy  audits  with  respect  to  wound  healing.  In  the  initial 
audits,  there  was  no  note  about  whether  the  abdominal  wound  was 
healing  in  42%  of  the  charts  examined.  By  the  time  of  the  reaudit, 
this  figure  had  dropped  to  6%,  an  improvement  of  36%.  Most  likely, 
however,  the  improvement  was  merely  a  matter  of  better  documentation 
since  it  is  unlikely  that  so  many  patients  previously  had  imperfect- 
ly-healed wounds. 

While  documentation  plays  an  important  role  in  the  continuity 
of  medical  care,  it  is  of  secondary  significance  in  terms  of  patient 
outcome . 

Non-Preventable  Outcomes.  By  definition,  non-preventable 
outcomes  are  not  directly  modified  by  changing  medical  practice. 
The  best  that  could  be  achieved  would  be  a  reduction  in  the  occu- 
pants of  this  class  as  a  result  of  improved  methods  at  other  points 
in  the  illness.  Such  outcomes  do  not  directly  stem  from  a  lapse  in 
physician  vigilance:  a  post-operative  hemorrhage,  for  example,  may 
occur  in  spite  of  good  surgical  technique.  Death  is  sometimes 
entirely  unpreventable ,  but  there  are  other  circumstances  in  which 
its  possibility  is  to  be  anticipated,  and  steps  could  be  taken  to 
avert  it.  Here  again,  there  is  a  connection  with  the  issue  of 
choice  of  criteria  (question  4  raised  earlier)  and  this  will  be 
further  discussed  under  that  heading.  Death  might  be  prevented,  for 
example,  if  the  patient  were  not  placed  at  risk  through  choice  of  a 
particular  procedure.  This  risk  might  be  a  function  of  age,  for 
example.  Such  criteria,  however,  were  rarely  if  ever  noted  to  have 
been  invoked,  or  if  invoked,  documented. 

Excluding  death,  only  two  non-preventable  outcome  criteria 
were  reaudited;  neither  showed  any  marked  change. 

The  chief  impression  gained  on  investigation  of  the  variation 
rates  of  both  preventable  and  non-preventable  outcomes  is  that 
little  if  any  significant  change  occurred.  Some  of  this  is  clearly 
due  to  a  lack  of  sufficient  reaudits.  That  in  itself  is  important, 
since  it  would  have  been  expected  that  serious  complications,  etc., 
would  be  included  in  the  criteria  for  reaudit.  Evidently  there  is 
room  for  a  lot  of  help  in  planning  audits  and  reaudits  by  individual 
hospitals.  One  useful  function  of  the  MCE  evaluation  could  be  to 
pinpoint  just  where  such  help  ought  to  be  offered. 

Mortality .  In  the  disorders  and  procedures  selected  for 
this  study,  a  standard  of  zero  mortality  was  used  in  the  audits.  In 
other  words,  no  one  should  be  expected  to  die  in  the  hospital  as  a 
result  of  these  illnesses.  For  elective  surgery,  the  zero  threshold 
is  appropriate.  For  myocardial  infarction,  it  is  unfortunately 
unrealistic.     The  fact  that  the  MCEs  in  this  sample  did  not  reduce 
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mortality  from  myocardial  infarction  does  not  indicate  a  failure  of 
the  MCE  process.  There  is  as  yet  no  way  of  setting  realistic 
standards  for  such  criteria,  but  it  is  clear  that  a  practical 
minimum  of  mortality  in  MI  will  be  closer  to  the  national  average  of 
19-20%  than  to  zero.  On  the  other  hand,  deaths  from  other  condi- 
tions or  procedures,  such  as  elective  operations,  are  far  less 
tolerable.  If  variation  rates  are  not  reduced  for  these  conditions 
and  procedures  through  the  MCE  process,  the  value  of  MCE  as  a  tool 
is  materially  reduced. 

For  the  three  medical  topics,  the  mortality  found  was  in  the 
same  range  as  that  found  by  the  National  Center  for  Health  Statis- 
tics, using  samples  of  hospital  inpatients.  This  provides  useful 
corroboration  for  the  accuracy  of  the  MCE  in  recording  the  death  of 
patients. 

Analysis  of  Question  2;        What  is  the  effect  of  justification 
of  variations  on  the  overall  influence  of  an  MCE? 

Many  criteria  are  designed  to  have  exceptions  explicitly  stated 
in  advance.  For  example,  abdominal  pain  is  a  criterion  of  appendi- 
citis, unless  the  patient  is  a  known  heroin  addict.  The  advantage 
of  recognizing  this  exception  in  advance  is  in  avoiding  wasted 
motion  when  reading  the  charts,  as  well  as  not  attributing  any  blame 
to  the  physician  who  may  have  correctly  diagnosed  appendicitis 
despite  no  complaint  of  pain. 

In  contrast,  justification  allows  the  physician  to  defend 
his/her  actions  even  when  they  fall  outside  the  criteria,  provided 
the  spirit  of  the  criteria  have  been  met.  Depending  on  the  evi- 
dence, some  variations  and/or  deficiencies  are  perhaps  excusable. 
Unlike  the  explicit  nature  of  the  predetermined  exceptions,  the 
justification  process  is  often  ad  hoc,  implicit,  even  arcane.  For 
these  reasons,  as  stated  earlier,  the  variation  rates  used  in  this 
study  were  calculated  from  pre justified  data. 

After  examining  the  individual  audit  minutes  for  a  few  cases  in 
which  a  death  had  to  be  explained,  further  comment  is  possible  on 
the  justification  process.  Two  cases  in  particular  serve  as  exam- 
ples. One  patient  who  died  after  a  cholecystectomy  had  developed 
cardiac  failure  secondary  to  a  post-operative  infection.  The 
committee  accepted  the  explanations  about  the  patient's  course,  and 
the  death  was  "justified."  Another  patient  had  an  appendectomy  with 
severe  complicating  infection.  This  patient  died,  and  the  records 
indicate  that  while  the  use  of  antibiotics  had  met  the  minimal 
standards  declared  by  the  terms  of  the  audit,  the  patient  had 
actually  been  significantly  mismanaged.  Nevertheless,  the  death  was 
"justified,"  and  there  is  no  record  of  any  attempt  to  advise  the 
surgeon  about  using  adequate  doses  of  antibiotics.  It  is  entirely 
possible  that  both  these  deaths  were  preventable.     A  serious  problem 
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in  trying  to  decide  what  to  do  about  these  deaths  is  of  course  the 
threat  of  malpractice.  If  the  families  had  decided  to  sue  these 
surgeons,  the  existence  of  an  adverse  decision  by  a  peer  review 
committee  would  enhance  the  family's  case.  Most  physicians  doubt 
that  the  committee  records  could  remain  inviolable,  regardless  of 
how  many  statements  are  made  to  the  contrary.  There  is  no  easy  way 
out  of  this  dilemma.  At  the  very  least,  the  MCE  would  have  been 
more  effective  if  the  committee  had  recommended  some  advice  or 
instruction  for  the  surgeon.  Internal  surgical  mortality  confer- 
ences do  not  seem  to  be  attended  by  the  same  fears.  A  capable 
surgical  director  would  not  hesitate  to  challenge  a  surgeon  on  his 
staff  and  even  restrict  privileges  until  the  problem  had  been 
properly  handled. 

The  escape  clause  of  justification  in  the  MCE  process  appears 
to  be  essential,  but  because  of  physicians'  distrust  of  programs 
imposed  by  the  government,  it  may  continue  to  be  abused.  Given  the 
latitude  allowed  in  the  justification  process,  it  appears  reasonable 
that  research  concerning  VRs  include  justified  as  well  as  unjusti- 
fied exceptions. 

Analysis  of  Question  3 :  What  are  the  implications  of  criteria 
selection? 

Two  conditions  must  be  satisfied  before  criteria  selection  can 
have  an  optimal  impact  on  the  MCE  process.  One  is  to  ensure  that  it 
is  possible  that  the  group  of  patient  records  selected  contains 
patients  with  the  condition  being  studied.  This  is  a  prerequisite. 
The  other  is  to  choose  criteria  which  will  distinguish  between 
effective  and  less  effective  care,  for  the  purposes  of  the  audit. 
These  two  functions  may  be  divergent.  Because  MCE  has  only  recently 
been  broadly  disseminated,  many  medical  staffs  have  not  yet  per- 
ceived this  problem.  They  choose  criteria  by  their  clinical  rele- 
vance, such  as  the  most  practical  way  of  establishing  the  diagnosis. 

Apart  from  criteria  which  relate  to  the  correctness  of  diagno- 
sis, the  remaining  process  criteria  for  management  and  treatment 
offer  a  broad  variety  of  possibilities  and  so  are  harder  to  select. 
Similarly,  choosing  the  significant  outcome  criteria,  other  than 
death,  presents  difficulties.  The  specialty  societies  have  produced 
many  criteria  sets  which  reduce  some  of  the  uncertainty,  but  they 
tend  to  be  more  effective  from  the  clinical,  descriptive  standpoint 
than  from  the  key  function  of  the  audit,  which  is  discrimination. 
Probably  the  majority  of  hospital  staffs  rely  on  the  AMA  criteria 
sets,  composed  of  the  individual  specialty  society  lists.  The 
result  is  that  most  of  the  audits  forming  the  basis  of  the  current 
study  have  adequate  clinical  validity,  but  are  less  adequate  for 
actually  measuring  care. 
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An  example  is  the  mortality  criterion  which  is  commmon  to 
the  three  surgical  topics.  Two  individual  deaths  were  briefly 
analyzed,  and  both  appeared  to  be  due  to  at  least  potentially 
preventable  causes.  For  the  patient  who  went  into  cardiac  failure, 
the  risk  might  have  been  suspected  because  of  his  age  or  presence  of 
known  heart  disease.  The  audits  in  this  sample  do  not  ask  this  type 
of  question.  It  turned  out  that  no  audits  had  considered  the 
patient's  risk  from  anaesthesia  before  surgery.  Some  audit  commit- 
tees had  expected  that  the  chart  show  that  hemoglobin,  chest  X-ray 
and  EKG  had  been  done  preoperatively ,  but  none  had  shown  any  concern 
for  the  results  of  the  tests. 

For  emergency  operations,  it  is  fair  to  waive  such  require- 
ments. For  elective  cases,  the  anaesthesiologist  or  the  patient's 
own  physician  in  most  accredited  hospitals  makes  a  note  before 
surgery,  and  includes  some  comment  on  the  patient's  underlying 
medical  condition.  The  decision  to  defer  surgery  based  on  an 
appraisal  of  risk  might  prevent  more  deaths  than  technically  ade- 
quate surgery.     Yet  none  of  the  MCEs  sampled  used  such  criteria. 

MCEs  could  improve  practice  by  insisting  that  patient  risk  be 
included  in  all  surgical  audits.  This  could  either  be  done  in  chart 
selection,  where  only  patients  of  roughly  similar  risk  status  are 
entered  into  the  sample,  or  it  could  be  done  in  the  audit  proper. 
Similarly,  serious  risks  inherent  in  patients'  use  of  drugs,  alco- 
hol, and  tobacco  ought  to  be  included.  Coming  to  terms  with  such 
questions  will  not  be  easy.  For  one  thing,  MCEs  have  only  grudg- 
ingly gained  acceptance.  The  concentration  on  group  behavior  when 
exploring  divergence  from  criteria  provides  anonymity  for  individual 
physicians.  Once  audits  start  to  probe,  for  example,  into  a  sur- 
geon's reasons  for  operating  on  a  patient  who  was  known  to  be  a  poor 
risk  in  advance,  they  can  become  very  threatening.  Yet  this  is  the 
way  in  which  maternal  mortality  figures  were  sharply  reduced,  when 
medical  societies  appointed  committees  to  investigate  every  single 
case  of  a  woman  dying  in  childbirth.  The  doctor  was  questioned 
directly  and  if  he  had  made  a  mistake,  it  could  not  be  hidden.  The 
main  difference  between  this  effective  procedure  and  the  MCE  proce- 
dure may  be  that  the  former  involves  a  professional  peer  committee 
appointed  internally  and  the  other  is  sponsored  by  the  Government. 
In  any  event,  if  criteria  are  to  be  effectively  employed,  more 
attention  must  be  paid  to  individual  cases.  This  activity  could 
link  analysis  criteria  more  closely  to  clinical  outcomes. 

The  statistical  analysis  of  variation  rates  considered  only  the 
most  common  criteria  found  in  the  sampled  MCEs.  Yet  each  audit  in  a 
topic  category  had  its  own  set  of  criteria,  chosen  by  the  audit 
committee.  Rarely  were  any  two  criteria  sets  alike.  Some  of  the 
criteria  which  appeared  in  a  minority  of  criteria  sets  were  judged 
to  be  unrealistic  or  irrelevant.  Though  some  of  these  criteria 
showed  high  variation  rates,   they  clearly  had  little  relation  to 
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quality  of  care.  One  such  example  was  the  requirement  that  every 
patient  subjected  to  appendectomy  have  a  serum  triglyceride  level 
test. 

Criteria  which  were  only  chosen  by  one  or  two  hospitals  and  did 
not  merit  a  full  calculation  of  variation  rates  for  audit  and 
reaudit  may  turn  out  to  be  important  in  that  they  suggest  useful 
criteria  for  future  MCEs.  One  such  criterion  was  an  upper  time 
limit  on  the  duration  of  a  cholecystectomy  operation.  A  priori/ 
there  is  reason  to  suppose  that  patients  do  best  when  anaesthetized 
for  the  shortest  possible  time.  Hence,  this  criterion  seems  to  have 
good  discriminating  potential  in  assessing  outcome.  Unduly  pro- 
longed operating  time  may  be  equated  with  a  higher  incidence  of 
undesirable  complications.  It  is  a  variable  which  can  be  measured 
and  so  seems   to  be   a  scientifically  valid  way  of  looking  at  care. 

A  more  purely  clinical  criterion  which  yet  has  some  discrimina- 
tive value  was  found  in  the  appendectomy  series.  One  hospital 
required  the  date  of  the  last  menstrual  period  in  women  of  child- 
bearing  age.  The  importance  lies  in  distinguishing  appendicitis 
from  ectopic  pregnancy.  Even  if  the  diagnosis  remains  appen- 
dicitis, it  is  critical  to  know  whether  the  patient  is  in  the 
vulnerable  state  of  early  pregnancy. 

Analysis  of  Question  4;  Are  topics  for  reaudits  appropriately 
chosen? 

If  the  hospitals  or  PSRO  were  to  adhere  strictly  to  the  regu- 
lations on  audit,  seven  out  of  ten  audits  should,  in  fact,  be 
reaudited.  To  date,  the  policy  on  audits  has  not  been  stringently 
applied.     Hence,   the  issue  of  choice  of  reaudits  deserves  mention. 

It  was  hard  to  find  a  pattern  in  the  sample  of  studies  ex- 
amined. Both  extremes  were  found;  i.e.,  some  topics  were  reaudited 
whether  compliance  had  been  excellent  or  very  poor.  Reading  the 
minutes  of  the  audit  committees  did  not  often  explain  this  diver- 
sity. 

Similarly,  when  a  topic  was  reaudited,  it  was  hard  to  find  a 
consistent  thread  in  the  choice  of  criteria.  In  the  series  on 
myocardial  infarction,  for  example,  practically  no  hospitals  checked 
on  the  incidence  of  arrhythmias  in  the  reaudits.  This  negates  the 
value  of  having  insisted  that  more  patients  be  admitted  to  coronary 
care  units  as  a  way  of  monitoring  for  dangerous  arrhythmias. 

In  some  hospitals,  the  followup  was  very  logical.  To  reduce 
the  number  of  patients  having  appendectomy  without  evidence  of 
appendicitis  (excluding  the  "incidental"  appendectomy),  one  hospital 
reexamined  a  hundred  charts  to  find  how  many  had  a  positive  path- 
ology report.     Another  audit  committee  was  concerned  about  patients 
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with  pneumonia  receiving  antibiotics  without  a  bacteriological 
culture  of  their  sputum.  The  subject  was  carefully  reaudited,  and 
led  to  a  diminution  in  the  number  of  such  cases. 

5.4    Summary  of  Quality  Assurance  Activities 

The  composite  picture  provided  by  the  MCE  implementation  and 
impact  studies  is  one  of  a  quality  assurance  mechanism  in  an  imter- 
mediate  stage  of  development.  Beyond  that,  the  following  conclu- 
sions may  be  stated  based  on  the  evidence  to  date: 

•  Initial  audits  are  being  performed  and  reported  with  in- 
creasing frequency  by  PSROs  and  delegated  hospitals.  Both 
the  total  numbers  of  MCE  audits  performed  and  the  average 
number  of  audits  reported  per  PSRO  are  growing  each  quarter. 
Over  22,000  initial  audits  have  been  reported  to  HSQB  in  the 
first  three  years. 

•  PSROs  vary  widely  in  the  number  of  initial  audits  they 
report  to  the  PMIS.  Some  are  consistently  productive,  while 
others  report  few  MCEs.  Though  it  is  possible  that  the 
latter  group  of  PSROs  are  carrying  out  studies  which  remain 
unreported,  they  are  not  showing  evidence  that  they  are 
fulfilling  their  mandate  to  monitor  the  quality  of  care 
provided  to  Federal  beneficiaries. 

•  MCE  topics  are  concentrated  in  traditional  clinical  areas: 
91%  of  initial  audits  focus  on  medical  diagnoses  or  surgical 
procedures,  with  the  remaining  9%  being  done  on  administra- 
tive and  miscellaneous  topics. 

•  The  most  frequent  MCE  topics  are  those  which  account  for  a 
large  proportion  of  admissions  and  patient  days.  The 
ranking  of  most  frequent  topics  is  fairly  stable.  The 
proportion  of  initial  audits  done  in  the  twenty  most  fre- 
quent topic  categories  appears  to  be  declining  over  time, 
suggesting   a   trend   toward   diversification   of  MCE  topics. 

•  Reaudits,  one  of  the  principal  increments  of  the  PSRO 
program  over  the  traditional  medical  audit  mechanism,  are 
not  being  reported  with  the  frequency  required  by  the 
findings  of  the  initial  audits. 

•  Data  of  sufficient  validity  to  assess  the  implementation  of 
areawide   MCEs   are  not  yet  available   through  the  PMIS. 

•  Among  a  nonrepresentative  sample  of  MCEs  which  had  completed 
the  full  audit/reaudit  cycle,  variations  from  locally-set 
standards  of  good  care  apparently  were  reduced  from  the  time 
of  audit  to  the  time  of  reaudit  for  criteria  involving  the 
medical  care  process.  Patient  outcomes  judged  to  be  pre- 
ventable exhibited  less  pronounced  reductions,  while  non- 
preventable  outcomes  showed  no  changes. 
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•  Clinical  assessment  of  variation  rate  changes  from  audit  to 
reaudit  generally  supported  the  validity  of  the  variation 
rate  methodology  and  findings,  though  in  some  cases  more 
complete  information  was  desired  than  was  available  in  the 
MCE  documentation. 
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6.       PROGRAM  MANAGEMENT 


6*1  Introduction 

The  purpose  of  this  section  is  to  address  some  of  the  eval- 
uation questions  concerning  the  organizational  aspects  of  the 
program.     Specifically,  the  Program  Evaluation  Plan/1  asked 

1.  How  effectively  and  efficiently  are  the  various  PSROs 
organized  in  terms  of  their: 

relationship  with  other  organizations, 

staffing, 

delegation  of  functions, 

review  activity  reimbursement  mechanism,  and 
review  activity? 

2.  What  has  been  the  effectiveness  of  PSRO  self-evalua- 
tions? 

3.  What  impact  have  organizational  and  environmental  fac- 
tors had  on  changes  in  physician  and  hospital  behavior? 

In  addition,  this  chapter  presents  a  description  of  PSRO 
program  performance  to  date  in  order  to  (1)  provide  a  context 
for  the  aggregate  hospital  utilization  analyses  and  cost  analyses, 
(2)  facilitate  understanding  of  the  processes  involved  in  PSRO 
implementation,  and  (3)  identify  factors  which  affect  the  ability 
of  PSROs  to  achieve  hospital  cost  and  quality  control  objectives. 

6.1.1     Studies  and  Data  Sources 

A  variety  of  sources  of  PSRO  program  information  are  employed. 
As  shown  in  Table  68,  these  include:  management  assessment  reports, 
project  officer  ratings,  and  data  from  the  PSRO  Management  Infor- 
mation System  (PMIS). 


/1  U.S.  DHEW,  Office  of  the  Assistant  Secretary  for  Health, 
Office  of  Professional  Standards  Review,  Program  Evalua- 
tion Plan:  Professional  Standards  Review  Organizations , 

September  22,  1975. 
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Data  derived  from  these  sources  are  used  in  this  section  to 
provide  the  following  studies: 

•  Analyses  of  Management  Assessments 

•  Analyses  of  Interrelationships  Among  Program  Processes 
and  Impact 

•  Preliminary  Analyses  of  Project  Officer  Ratings  of 
PSRO  Performance. 


6.1.2  Constraints 

The  discussions  of  the  PSRO  process  and  performance  data 
presented  here  are  limited  to  the  PSROs  under  study  and  cannot 
be  generalized  to  the  total  population  of  PSROs.  The  project 
officer  ratings  include  107  PSROs  and  the  management  assessments 
cover  26  of  those  PSROs.  However,  it  is  possible  to  obtain  from 
these  sources  a  sense  of  the  nature  and  diversity  of  PSROs. 

6.2    Management  Assessments 

6.2.1  Background 

In  November  19  76,  the  Health  Standards  Quality  Bureau  ini- 
tiated visits  to  PSROs  in  order  to  assess  their  status  and  per- 
formance. Site  assessment  efforts  were  developed  to  examine  PSROs' 
implementation  efforts  as  they  progressed  from  conditional  to  full 
implementation  status,  to  identify  strengths  and  problems  in  in- 
dividual PSROs,  and  to  recommend  solutions  to  the  problems  identi- 
fied. 

In  developing  the  assessment  methodology,  HSBQ  focused  on 
establishing  a  system  that  could  measure  those  aspects  of  per- 
formance which  were  important  for  the  project  at  its  particular 
stage  of  development.  Since  it  was  not  feasible  to  measure  a 
PSRO's  impact  within  the  course  of  a  three-day  visit,  assessors 
concentrated  on  management,  organization,  and  review  develop- 
ments. Assessment  teams  were  usually  comprised  of  five  members 
from  both  Federal  and  non-Federal  agencies,  including  consider- 
able use  of  reviewers  from  other  PSROs.  Individual  team  members 
were  assigned  responsibility  for  particular  subject  areas.  In 
undertaking  reviews,  team  members  were  guided  by  standards  cover- 
ing the  role  of  the  board  of  directors?  program  management;  ex- 
ternal relations;   technical  functions  including  concurrent  review, 
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medical  care  evaluation  studies,  profile  analysis,  and  continuing 
medical  education;  data  management;  and  impact.  Team  members  were 
required  to  complete  individual  narrative  reports  for  their  respec- 
tive areas*  These  were  incorporated  into  a  final  summary  report 
organized  by  the  site  assessment  coordinator  who  was  an  HSQB  staff 
member. 

The  reports  appeared  to  the  evaluators  as  an  appropriate  source 
for  an  initial  examination  of  PSRO  performance.  To  this  end,  three 
efforts  were  undertaken  to  cull  information  from  site  assessment 
materials.  These  efforts  included  anywhere  from  18-26  sites  de- 
pending on  the  availability  of  information  at  the  time  the  evalua- 
tion was  undertaken. 

One  analysis  of  eighteen  published  site  assessment  reports 
was  conducted  by  staff  of  the  University  Health  Policy  Consortium; 
the  other  two  analyses  were  conducted  by  a  consultant  to  HCFA  using 
twenty-six  ratings  completed  by  HSQB  site  assessment  coordinators 
and  twenty  published  site  assessment  reports.  Methodologies  for 
each  of  these  three  efforts,  as  well  as  the  data  limitations,  are 
presented  below.  Findings  from  the  three  analyses  are  combined  to 
provide  an  overall  view  of  PSRO  performance  at  the  time  of  assess- 
ment. 

6.2.2    Objectives  and  Methodologies 

6.2.2.1  Analysis  I;  Descriptive  Content  Analysis 

The  purpose  of  this  anlysis  is  to  aggregate  systematically 
the  site  assessment  data  by  transforming  the  prose  style  of  the 
final  site  reports  into  quantitative  measures,  thereby  making 
general  statements  about  the  ways  in  which  PSROs  are  managed. 
Eighteen  site  assessment  reports  were  reviewed  with  the  most 
complete  selected  for  analysis */\_  A  series  of  questions  was  devel- 
oped which  operationalized  salient  information  in  the  reports. 
These  questions  were  combined  in  an  instrument  referred  to  as 
the  "Site  Assessment  Protocol".  Two  abstractors  completed  the 
items  for  each  site  assessment  report. 

6.2.2.2  Analysis   II:   Site  Assessment  Coordinator  Ratings 

HSQB  site  assessment  coordinators  rated  the  performance  of 
26  sites  assessed  by  August  1978  using  an  instrument  consisting 
of  46  items.  This  instrument  was  designed  so  that  coordinators 
could  give   one   of   five  ratings—poor ,   fair,   average,    good  or 


/1  Eighteen  site  assessment  reports  were  made  available  to  the 
investigators;  eight  additional  assessments  had  not  been 
completed  at  the  time  of  analysis. 
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excellent  on  each  item.  Ratings  for  most  of  the  items  were  normally 
distributed.  As  might  be  expected,  ratings  were  in  general  agree- 
ment with  those  performed  by  the  abstractors  in  the  content  analy- 
sis. 


6.2.2.3  Analysis  III;   Strengths  and  Weaknesses  Analysis 

Site  assessors  identified  specific  strengths  and  weaknesses 
for  the  sites  visited.  These  were  documented  in  the  site  assess- 
ment reports.  In  this  analysis,  specific  strengths  and  weaknesses 
were  catalogued. 


6.2.3    Data  Limitations 

Before  presenting  the  findings  from  these  analyses,  the 
reader  should  be  cautioned  about  some  aspects  of  the  site  assess- 
ments themselves  which  affect  the  nature  of  the  findings: 

(1)  Site  assessments  were  designed  as  management  tools  and 
not  for  evaluation  purposes.  Although  systematic  attempts 
were  made  to  collect  comparable  information  from  each  PSRO 
reviewed,  the  reports  contain  disparities  with  respect  to 
both  the  quality  and  amount  of  detail  paid  to  like  issues 
across  PSROs. 

(2)  The  site  assessment  teams  began  their  operations  in 
November  1976,  and  at  present  continue  to  make  site  in- 
vestigations. Thus,  the  early  reports  were  completed 
almost  two  years  ago,  while  some  of  the  more  recently 
completed  assessments  are  barely  four  months  old.  Sim- 
ilarly, many  of  the  PSROs  under  analysis  began  imple- 
mentation at  differing  times;  thus,  they  were  assessed  at 
differing  stages  in  their  respective  life-cycles.  The 
reader  should  also  be  cautioned  that  many  of  the  defici- 
encies identified  have  subsequently  been  corrected  (See 
section  6.2.5  for  discussion).  While  the  data  presented  in 
the  site  assessment  analysis  are  not  necessarily  a  reflec- 
tion of  the  current  performance  levels  of  assessed  PSROs, 
there  is  a  high  rank  order  correlation  among  the  scores 
assigned  by  the  site  assessment  coordinators  and  the  1978 
project  officer  ratings. 

(3)  The  selection  of  sites  to  be  analyzed  was  not  based  on 
any  random  or  stratified  sampling  procedures.  Therefore, 
statements  about  PSROs  deriving  from  the  analyses  can 
not  be  generalized  to  the  large  uninvestigated  PSRO  popula- 
tion. 
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(4)  Although  site  assessors  were  health  care  professionals, 
they  were  not  specifically  trained  for  assessment  purposes, 
nor  was  any  effort  carried  out  to  establish  a  level  of 
"inter-assessor  reliability"  (i.e.,  the  extent  to  which 
assessors  would  agree  on  the  quality  of  performance  of  a 
given  PSRO  function  or  review  procedure). 

(5)  Before  the  site  assessment  reports  were  disseminated,  PSROs 
were  given  the  opportunity  to  both  comment  upon  and  redress 
statements  within  the  evaluations.  The  magnitude  of  PSRO 
interaction  in  the  preparation  of  the  final  reports  is 
quite  varied  and,  more  often  than  not,  unknown. 

In  addition  to  the  problems  just  described,  which  were 
imposed  on  the  investigators  by  the  nature  of  the  site  assessments, 
no  attempts  were  made  in  the  three  efforts  to  measure  inter-coder 
reliablity,  due  to  time  limitations. 


6.2.4  Findings 


6.2.4.1     Summary  of  Findings 

(1)  Two-thirds  of  the  PSROs  assessed  had  review  programs  before 
becoming  PSROs.  This  prior  experience  appears  to  have  had 
beneficial  consequences  for  implementation  of  PSRO  review. 
However,  a  plateau  of  performance  appears  to  be  reached  in 
concurrent  review  activities,  with  the  older  PSROs  receiv- 
ing scores  only  slightly  higher  than  the  less  experienced 
PSROs.  Other  areas  of  review  do  appear  to  continue  to 
improve  over  time.  This  is  particularly  true  in  terms  of 
variations  in  hospital  review  and  other  innovative  review 
activities. 


(2)  Assessors  were  complimentary  about  the  competence,  dedic- 
ation and  knowledge  of  PSRO  executive  boards. 

(3)  In  these  PSROs,  about  90  percent  of  board  members  were 
physicians. 

(4)  Although  committee  structure  was  found  to  vary  greatly  from 
PSRO  to  PSRO,  all  of  the  major  committee  functions  were 
performed. 


(5)  Assessors  viewed  PSRO  staff  as  competent;  however,  they 
noted  a  consistent  lack  of  specific  training  for  many  staff 
positions,  such  as  physician  advisors,  nurse  coordinators 
and  board  members. 
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On  the  other  hand,  PSROs  tend  to  receive  low  scores  on: 

(1)  withdrawal  of  delegation  from  hospitals, 

(2)  items  dealing  with  MCE  studies,  including  percentage  of 
required  MCEs  performed,  number  of  areawide  and  hospital- 
specific  topics,  percentage  of  required  reaudits  performed 
in  relation  to  initial  MCEs,  and  deficiencies  identified  in 
MCE  studies 

(3)  ancillary  services  review, 

(4)  review  of  physician  services,  and 

(5)  items  related  to  denials  and  sanctions,  including  number  of 
denials  issued,  proceedings  for  potential  sanctions  initi- 
ated against  a  practitioner  or  hospital,  actual  recommen- 
dation for  sanction  forwarded  to  the  Secretary,  and  state 
agency  and  DHEW  notified  of  potential  fraud  or  program 
abuse. 

As  stated,  major  differences  were  found  in  the  ways  PSROs 
were  managed.  There  were  a  series  of  "high-problem  areas"  which 
are  common  to  all  PSROs,  whether  they  are  "well-managed"  or  not* 
Some  of  these  include:  monitoring  of  impacts,  monitoring  of  cost- 
effectiveness,  and  a  lack  of  formal  training  for  key  personnel  and 
coordination  of  physician  advisor  and  review  coordinators.  Detailed 
discussions  of  these  areas  appear  below. 

The  overall  picture  of  the  high  performing  PSROs  emerging 
from  the  strengths  and  weaknesses  analysis  is  that  of  organizations 
in  which  board,  staff,  and  the  physician  membership  are  highly 
committed  to  the  organization's  objectives  and  goals.  Whether 
considering  concurrent  review  activities  or  linking  review  with 
continuing  education,  the  evaluators  consistently  mention  staff, 
physician,  or  board  support  as  a  strength  in  the  better  performing 
PSROs.     Staff  competence  in  many  areas  is  also  often  cited. 

The  data  indicate  there  was  not  a  great  deal  of  difference 
in  the  specific  problems  uncovered  for  both  the  lower  rated  and 
higher  rated  PSROs.  Although  these  PSROs  had  accomplished  a  great 
deal  at  the  time  of  their  assessments,  considerable  latitude 
existed  for  improvement. 

In  an  attempt  to  rate  the  performance  of  PSROs  in  key  areas, 
questions  on  the  coordinator  rating  instrument  were  grouped  into 
eight  categories.  Responses  to  individual  items  were  scored 
from  0-4  points.     Scores  for  all  items  within  a  given  category  were 
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then  computed;  and  a  total  score  representing  the  sum  of  the  eight 
areas  was  calculated.  Average  total  scores  for  each  category  and 
for  all  categories  combined  are  presented  in  Table  69. 


TABLE  69 

COORDINATOR  ASSESSMENT  OF  PSRO  PERFORMANCE  IN 
EIGHT  AREAS  (26  PSROs) 


Range  of 

Actual 

Areas 

Possible 

Range  of 

Average 

Scores 

Scores 

Score 

Organizational  Strength 

0-28 

9-23 

15 

General  Management 

0-35 

3-29 

20 

External  Relations 

0-16 

6-15 

10 

Concurrent  Review 

0-23 

3-19 

12 

MCEs 

0-20 

0-14 

4 

Other  Review 

0-10 

0-7 

2 

Denials ,  Sanctions , 

Fraud,  Abuse 

0-16 

0-9 

1 

Responsiveness  of 

Review 

0-27 

0-23 

12 

TOTAL  SCORE 

0-175 

28-1 16 

76 

Table  69  demonstrates  that  outstanding  ratings  were 
not  achieved  in  any  of  the  components.  Average  ratings  were  partic- 
ularly low  in  the  areas  of  other  review  and  denials.  Most  PSROs, 
however,  achieved  relatively  high  ratings  in  the  areas  of  organiza- 
tional strength  and  general  management.  Ratings  for  most  categories 
were  normally  distributed  with  the  exception  of  the  MCE,  other 
review,  and  sanction/denial  areas,  all  of  which  were  skewed  toward 
the  lower  ratings.  Each  of  these  eight  areas  is  discussed  separ- 
ately below. 

6.2.4.2    Specific  Findings 

A.    Organizational  Strength 

This  area  was  measured  on  the  coordinator  rating  questionnaire 
by  seven  questions  including  level  of  physician  involvement  (includ- 
ing percentage  of  community's  physicians  who  are  members  of  the 
PSRO,  physicians'  participation  in  PSRO  activities),  leadership  and 
stability  of  the  governing  body,  and  particiaption  in  committees. 
PSROs  tended  to  score  relatively  high  in  this  area,  with  a  rela- 
tively small  spread  between  those  scoring  highest  and  those  scoring 
lowest. 
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( 1 )     Executive  Boards 


The  site  assessment  team  reports  usually  contained  some 
laudatory  comments  about  PSRO  executive  boards.  Board  members  are 
considered  highly  competent,  dedicated  and  knowledgeable.  Many 
board  members  have  occupied  leadership  positions  in  previous  review 
organizations,  while  others  were  early  supporters  of  the  PSRO  review 
concept.  Board  members  are  often  pivotal  actors  in  the  larger 
medical  community.  While  the  composition  of  PSRO  boards  varies 
slightly,  almost  90  percent  are  physicians.  Health  care  practi- 
tioners other  than  physicians  are  generally  excluded  from  board 
positions,  though  some  do  serve  on  committees. 

Half  of  the  site  assessment  teams  reported  that  the  executive 
director  generally  serves  as  the  focal  point  for  all  decision-making 
efforts.  The  executive  board  members  were  frequently  cited  as  only 
marginally  involved  in  the  decision-making  process,  with  a  few 
involved  in  a  cursory  pro  forma  fashion. 

As  indicated  below,  the  several  assessment  teams  were  critical 
in  their  evaluations  of  the  effectiveness  of  the  PSRO  boards: 


"How  effectively  does  the  Board  oversee  operations?" 

Very  effectively  (n=5) 

Somewhat  effectively  (n=4) 

Not  very  effectively  (n=3) 

Ineffectively  (n=1) 

No  discussion  of  board  effectiveness  (n=5) 


A  general  lack  of  board  control  over  operational  affairs  was 
cited  as  a  problem  in  four  site  reports.  One  example  of  this 
problem  emerged  in  discussions  of  the  budgetary  process.  Of  the 
nine  site  reports  which  discussed  levels  of  board  involvement  in 
fiscal  matters,  six  judged  the  board  to  be  either  totally  uninvolved 
or  only  marginally  involved  in  the  setting  of  budgets. 

In  eighteen  site  assessments  examined,  it  was  noted  that 
nine  PSRO  boards  enunciated  measurable  objectives  for  the  PSRO,  but 
six  did  not.  However,  eleven  site  teams  recommended  that  boards 
clearly  define  their  own  roles  and  responsibilities;  The  lack  of 
understanding  of  internal  board  organization  was  cited  in  a  number 
of  site  reports,  though  this  criticism  was  not  elaborated  upon.  It 
was  presumed  to  have  deleterious  effects  on  the  operations  of  the 
PSROs. 


(2)  Committees 


Committee  structure  varies  greatly  from  PSRO  to  PSRO,  although 
all  major  functions  are  performed.  Committees,  however,  were  often 
seen  by  the  site  teams  as  weak  and  uneven,  their  activities  perfunc- 
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tory,  and  their  policy  inputs  purely  "advisory"  in  nature.  Two  PSRO 
areas  deviated  from  this  consistent  pattern,  with  highly  involved 
and  vigorous  committees  which  initiate  major  policy  changes. 

The   overall   assessment   of   the   effectiveness   of   the  major 
committees  was  uneven,  as  the  following  indicates: 

Effectiveness  of  Major  PSRO  Committees; 


Outstanding  (n=2) 

Good  (n=6) 

Fair  (n=4) 

Poor  (n=3) 

Not  discussed  (n=3) 


(3)     Physician  Membership  and  Participation 

Physician  membership  levels  varied  among  the  eighteen  assess- 
ments, ranging  between  38  percent  and  85  percent,  with  ten  above  60 
percent  and  eight  below.  While  the  mean  percentage  of  physicians 
belonging  to  the  assessed  PSROs  is  slightly  higher  than  the  national 
average  (53%),  there  were  multiple  indicators  that  membership  itself 
cannot  be  equated  with  physician  participation.  Some  of  these 
indicators  are  discussed  below. 

Physician  inputs  from  the  community  at  large  were  judged 
by  the  site  teams  as  minimal,  though  outright  resistance  to  PSRO 
activity  was  not  discernible.  Of  the  thirteen  site  reports  which 
discussed  physician  recruitment  efforts,  eleven  indicated  that  the 
PSROs  were  actually  soliciting  the  involvement  of  physicians  from 
local  communittees . 

The  teams,  in  general,  viewed  physician  participation  in  review 
activities  as  important.  The  degree  of  physician  participation  and 
commitment,  however,  was  seen  by  the  site  teams  as  varied.  Of  the 
ten  reports  which  discussed  the  level  of  physician  participation  in 
on-site  monitoring  visits,  five  indicated  that  physicians  were 
"very  involved,"  two  that  physicians  were  "somewhat  involved,"  and 
three  that  physicians  were  not  involved  at  all. 

Table  70  displays  the  specific  strengths  and  weaknesses 
identified  by  site  assessors  concerning  the  board  of  directors, 
PSRO  committees,  and  physician  involvement. 

B.    General  Management 

In  the  area  of  general  management,   the  coordinator  rating 
questionnaire   included  competence   of   executive   and  administrative 
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TABLE  70 
ORGANIZATIONAL  STRENGTH 
NUMBER  OF  TIMES  IDENTIFIED  AS  A 
STRENGTH  OR  WEAKNESS 


Strength  Weakness 

Board  dedication  1  1 

Board  involvement  10  2 

Staff  support  3  2 

Goal  setting  5  2 

Cost  consciousness  1  4 

Physician  support  2  0 

Representation  5  1 

Board  leadership  3  1 

Role  of  Executive  Director  1  1 

Training  0  7 

Committee  structure  4  4 

Membership  4  2 

Specific  committees  2  2 

Understanding  of  responsibilities  0  3 

Physician  suport  and  involvement  2  1 

Functioning  3  2 

Physician  support  and  involvement  9  3 

Recruitment  approach  3  4 

••  Membership  size  5  1 

Voluntary  efforts  3  0 

Committees  3  3 

Involvement  7  7 

Communication  3  1 

Review  involvement  6  5 


staff,  staff  stability,  and  data  management.  There  was  a  great  deal 
of  variation  among  scores  in  this  category.  On  the  whole,  PSROs 
scoring  high  in  organizational  strength  also  tended  to  score  high  in 
general  management. 

(1)  Staffing 

Among  the  eighteen  assessments,  PSRO  office  staffing  was 
universally  viewed  as  very  qualified,  with  personnel  turnover 
creating  problems  in  only  two  PSRO  areas.  There  were  three  in- 
stances of  shortages  cited  in  management  staffing,  while  over- 
staffing  was    reported   in   one   PSRO.       The   site   reports,  however, 
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did  posit  that  problems  occurred  as  a  result  of  the  lack  of  formal 
training  given  to  some  staff  members.  Of  the  thirteen  reports  which 
treated  this  subject,  nine  noted  that  formal  training  was  absent  and 
was  contributing  both  to  a  general  sense  of  confusion  concerning  job 
roles  and  the  delineation  of  job  responsibilities.  The  quality  of 
training  given  was  judged  as  excellent  in  four  PSROs,  good  in  three 
and  fair  in  one. 


(2)     Staff  Communication 


Site  teams  viewed  communications  between  the  board  and  its 
committees,  and  between  committees  and  staff  as  ranging  from  "good" 
to  "poor."  Communication  between  the  Executive  Director  and  his  or 
her  staff  was  judged  positively. 


Evaluation  of  Quality  of 
Communication  between  . . . 

No 

Good  Acceptable  Poor  Mention 

Board  and  its  committees                (n=6)  (n=1 )            (n=5)  (n=6) 

Committees  and  staff                        (n=5)  (n=1)            (n=4)  (n=8) 

Executive  Director  and  staff         (n=10)  (n=2)            (n=9)  (n=6) 


Table  71  displays  the  specific  strengths  and  weaknesses 
identified  by  site  assessors  in  general  management,  including 
staffing,  operations  and  organization,  financial  management,  and 
data  management. 

C.    External  Relations 

This  area  was  measured  on  the  coordinator  rating  question- 
naire by  four  questions  measuring  level  of  agreement  with  Medicare 
and  Medicaid  decisions,  and  interactions  with  Health  Systems  Agen- 
cies. PSROs  tended  to  have  generally  average  relations  with  other 
organizations.  Scores  in  this  area  tend  not  to  be  related  to 
organizational  strength  and  general  management.  For  example,  one  of 
the  highest  scorers  in  this  category  achieved  relatively  low  ratings 
in  the  other  two  categories.  A  general  discussion  of  PSROs'  rela- 
tionships with  other  organizations  follows. 

A  number  of  PSROs  were  originally  sponsored  by  local  medical 
societies.  Many  of  the  more  committed  board  members  had  previously 
assumed  leadership  positions  in  these  societies,  and  continue  to 
maintain  strong  and  amicable  ties.  Because  of  prior  relations, 
communication  between  PSROs  and  medical  societies  often  is  on  an  ad 
hoc  or  informal  basis. 

One  of  the  characteristics  of  PSROs  with  a  large  number  of  area 
medical  schools  is  a  sharing  of  leadership  functions.     The  physician 
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TABLE  71 

GENERAL  MANAGEMENT:       STRENGTHS  AND  WEAKNESSES 

Number  of  Times 
Identified  As  A 
Strength  Weakness 

1 .  Staffing 

Staff  qualifications  and  experience 
Staff  dedication 
Staff  competence 
Staff  morale 
Executive  Director 
Personnel  Director 
Audit  Manager 

Staff  understanding  of  role 

and  responsibilities 
Role  of  Executive  Director/ 

Medical  Director 
Workloads  on  staff 
Staff  size 
Stability 
Recruitment 

2*     Operations  and  Organization 


Personnel  policies  4  9 

Operation  and  policy  0  4 

Organizational  structure  5  3 

Staff  development  and  training  3  3 

Line  staff  communication  5  2 

Management  4  0 

Delegation  of  authority  3  3 

Physician  involvement  3  0 

3.     Financial  Management 

Financial  management  system  8  2 

Financial  Manager  1  1 

Bookkeeper  2  0 

Status  of  books  2  2 

Documentation  2  0 

Staffing  2  5 

Routine  audits  3  0 

Expenditures  0  2 

Board  role  in  financial  affairs  1  3 

Accountability  0  2 

Status  of  cost  data  and  reporting  2  5 

Property  management  2  0 

Functional  reporting  3  1 

Training  2  1 

Budget  preparation  0  2 


7  1 

6  0 

6  1 

5  1 

4  4 

3  1 

0  3 

4  8 

1  4 
NA  4 

2  4 
2  1 
0  3 
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TABLE  71  (Continued) 
GENERAL  MANAGEMENT:        STRENGTHS  AND  WEAKNESSES 

Number  of  Times 
Identified  As  A 


Strength  Weakness 

Data  Management 

a.  General 

Board  involvement  3  1 

Committee  involvement  3  1 

Staff  1 0  2 

Management/Organization  4  2 

Training  6  6 

System  integration  4  4 

Coordination  with  hospitals  8  1 

Control  and  monitoring  8  2 

Plan  4  2 

System  design  9  3 

Implementation  6  3 

Duplication  0  2 

Timeliness  5  4 

Cost  of  4  2 

b.  Input 

Use  of  optional  elements  7  8 

Data  abstracts  9  5 

Coding  methods  7  1 

Re-abstracting  2  1 

Diagnostic  and  procedural  coding        1  10 

Quality  control  5  8 

Physician  and  patient  IDs  2  5 

Collection  and  routing  7  4 

c.  Processing 

Implementation  2  0 

Procurement  process  4  2 

Flexibility  2  4 

Subcontractor  4  5 

Design  0  2 

Abstract  backlog  NA  3 

d.  Output 

Provision  for  use  of  reports  6  8 
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leadership  of  many  medical  schools  is  actively  involved  in  PSRO 
affairs  either  through  board,  or  committee  membership.  Thus, 
relations  are  cordial,  and  communication  informal  but  effective. 

It  appears  that  although  relations  with  area  medical  schools 
are  quite  good,  the  typical  PSRO  itself  assumes  a  very  limited  role 
as  an  educational  medium.  It  does  not  often  generate  programs  of 
continuing  medical  education,  but  assumes  what  could  be  called  a 
passive  involvement  in  the  programs  generated  by  other  institu- 
tions, such  as  area  medical  schools. 

Relations  with  state  hospital  associations  are  cordial, 
though  the  quality  of  relations  with  individual  hospitals  is  quite 
varied.  While  top  level  hospital  administrative  staff  are  generally 
in  direct  contact  with  PSRO  executive  staff,  formal  channels  of 
communication  between  the  PSRO  executive  staff  and  hospital  middle 
management  are  weak.  Actual  hospital  resistance  to  PSRO  activities, 
however,  is  virtually  non-existent. 

In  general,  the  site  reports  provide  little  information  about 
PSRO  fiscal  agent  relations.  In  reports  where  information  on  this 
issue  is  provided,  relations  with  agencies  vary  from  cordial  to 
tense.  While  both  hospitals  and  payment  agencies  have  a  basic 
understanding  of  one  another,  reimbursement  disputes  do  occur  and 
relations  are  sometimes  strained. 

Table  72  displays  the  specific  strengths  and  weaknesses 
identified  by  site  assessors  concerning  external  relations. 


D.     Concurrent  Review 

This  area  was  measured  on  the  coordinator  rating  questionnaire 
by  six  questions  concerning  degree  of  implementation,  level  of 
monitoring  of  delegated  hospital  review,  degree  of  cooperation  with 
hospitals,  withdrawal  of  delegation,  and  training  provided.  A 
normal  distribution  of  scores  was  obtained.  Implementation  and 
operation  of  concurrent  review  systems  did  not  appear  to  be  related 
to  any  of  the  other  areas  discussed.  Performance  in  this  category 
appeared  to  be  a  function  of  age,  with  older  PSROs  achieving  higher 
performance  ratings. 

( 1 )     Implementation  of  Review 

A  top  priority  of  the  PSROs  has  been  to  get  binding  con- 
current review  (CR)  underway  in  as  many  hospitals  as  possible.  Of 
the  eighteen  examined,  seven  PSROs  had  fully  implemented  CR  in  all 
area  hospitals.     The  reports  indicate  that  this  priority  had  not 
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TABLE  72 

EXTERNAL  RELATIONS:       STRENGTHS  AND  WEAKNESSES 


Number  of  Times 
Identified  As  A 


a.  Hospitals-PSRO  Relationships 
Hospital  Association 
Hospitals 

Hospital  Rate  Commission 

b.  Fiscal  Intermediary 
General  Relationship 
Medicare  Intermediary 
Monitoring 

Denials 

c.  Medicaid 

General  Relationships 
Information  Sharing 

d.  Medicare 
Monitoring 

e.  Title  V 

General  Relationship 
f •      Nusing  Homes 

General  Relationships 

g.  Other  PSROs 
General  Relationship 

h.  Medical  Societies/Area 

Physicians 
General  Relationships 

i.  Planning  Agencies 
General  Relationships 
Joint  PSRO/HSA  Project 
PSRO/HSA  Agreements 
Data  Sharing 

j.      External  Relations  in  General 
Board  Involvement 
Communication 
Information  Sharing 
PSRO  Image/PR 
Physician  Involvement 
Problem  Resolution 
Strategy 
Staff  Use 
Advisory  Group 


Strength 

7 
4 

2 

6 
2 
1 
0 

6 
0 

1 

2 

4 

1 

11 

7 
2 
2 
1 

4 
4 
1 

2 
2 
2 
4 
3 
3 


Weakness 

1 
4 

0 

5 
1 
3 
3 

4 

2 

2 
3 
2 
1 


6 
0 
0 
4 

0 
3 
* 
1 
4 
1 
0 
0 
1 
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been  satisfactorily  achieved  in  all  PSRO  areas.  Some  of  the  cited 
problems  included:  review  procedure  inconsistencies  and  failure  to 
relate  the  objectives  of  CR  to  utilization  problems. 

(2)  Review  Services  Staff 

Executive  boards  were  found  to  have  expressed  a  genuine 
desire  to  fully  implement  a  series  of  systematic  review  programs. 
However,  a  clear  set  of  problems  has  been  recognized  with  respect 
to  review  services  staff.    They  are  as  follows: 

(a)  Assessment  teams  have  consistently  noted  that  review 
services  staff  are  confused  about  job  responsibilities.  While 
Review  Coordinators  (RCs)  have  the  appropriate  background  skills, 
training  programs  were  cited  in  only  half  of  the  PSROs.  Selection 
procedures  for  Physician  Advisors  (PAs)  were  judged  by  site  teams  as 
poor  in  four  of  six  areas  where  such  procedures  were  reviewed.  Of 
the  twelve  reports  which  assessed  quality  of  PA  training,  it  was 
noted  that  eight  areas  gave  no  formal  training  to  PAs. 

(b)  The  utilization  of  PAs  in  quality  assurance  functions 
was  seen  as  problematic  in  more  than  half  the  PSROs. 

(c)  Communication  between  PAs  and  RCs  is  often  inadequate,  and 
the  quality  of  physician  documentation  has  been  a  problem,  though  it 
has  improved.  The  site  reports  noted  that  in  some  PSRO  areas, 
hospital  abstracts  are  received  in  fragments,  causing  indecision  and 
confusion  among  RCs.  Excellent  PA-RC  relations  were  reported  in  one 
PSRO. 

(3)  Development  of  Norms  and  Criteria 

Well-established  norms  and  criteria  were  cited  as  exemplary  in 
three  PSRO  areas.  However,  site  assessment  teams  have  regarded  the 
establishment  of  areawide  uorms  and  criteria  as  a  general  problem. 
Specified  criteria  are  not  strictly  adhered  to,  nor  are  areawide 
norms  currently  being  utilized  in  fourteen  of  the  eighteen  PSROs.  A 
significant  number  of  PSROs,  however,  are  currently  attempting  to 
systematize  both  norms  in  use  and  criteria  sets. 

As  the  following  table  indicates,  the  teams  did  not  always 
favorably  judge  the  appropriateness  for  length  of  stay  (LOS)  norms 
in  use: 
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Level  of  Appropriateness  for  LOS  Norms  in  Use/1 


Very  appropriate  (n=1) 

Appropriate  ( n=3 ) 

Not  very  appropriate  (n=4) 

Inappropriate  ( n=4 ) 

Not  mentioned  (n=6) 


A  cross-tabulation  (see  below)  suggests  that  the  number  of 
years  PSROs  have  been  in  existence  is  related  to  the  presence  of  a 
formal  procedure  to  review  norms* 


Number  of  PSROs  That  Have  A  Formal  Procedure 
To  Review  Norms 


Procedure  to  Old  PSRO  New  PSRO 

Review  Norms?        (>  3  years)       (<  2  years) 


Yes  6  3 

No  1  3 


Criteria  are  modified  for  local  use  in  thirteen  of  the  eighteen 
PSROs /  while  the  site  assessment  reports  indicate  that  nine  PSROs 
employ  a  regular  procedure  to  review  both  norms  and  criteria  sets. 
These  criteria  sets,  however,  have  been  deemed  "difficult  to  use" 
in  eight  of  the  nine  areas. 

(4)    Profile  Analysis  and  Focused  Review 

Profile  analysis  had  been  implemented  in  six  of  the  eighteen 
PSRO  areas.  Profile  analysis  data  have  been  utilized  in  the  devel- 
opment of  norms,  in  monitoring,  and  in  assessing  the  quality  of 
regional  health  care. 

The  cross-tabulations  which  follow  suggest  that  both  focused 
review  and  profile  analysis  are  more  likely  to  be  operative  in 
the  older  PSROs. 


/I     The  site  reports  did  not  indicate  whether  LOS  norms  were  judged 
as  too  long  or  too  short. 
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Number  of  PSROs  With  Focused  Review 


Yes 
No 

Under 
Develop- 
ment 


Old 
4 

2 


New 
0 
2 


1 


2 


Number  of  PSROs  With  Profile  Analysis 


Yes 
No 


Old 
5 
5 


New 
1 
7 


( 5)    Monitoring  of  Review 

Very  effective  monitoring  was  cited  in  six  PSROs.  In  five 
PSROs,  questions  were  raised  as  to  the  adequacy  of  monitoring 
in  terms  of  sample  size,  frequency  and  completeness.  In  addition, 
only  half  of  the  PSROs  followed  up  on  issues  addressed  in  the 
monitoring  reports,  and  only  three  revised  procedures.  Eight 
reports  indicated  that  strengthening  of  monitoring  procedures  would 
be  beneficial  and  eight  recommended  increased  utilization  of 
existing  CR  data. 

Strengths  and  weaknesses  identified  by  site  assessors  in  the 
concurrent  review  area  are  presented  in  Table  73. 


E.     Medical  Care  Evaluation  Studies 

This  area  was  measured  on  the  coordinator  rating  question- 
naire by  four  questions  considering  the  percentage  of  required  MCEs 
performed,  number  of  areawide  studies  per  year,  appropriateness  of 
topics,  percentage  of  reaudits  required,  and  deficiencies  identi- 
fied. PSROs  tended  to  have  either  high  or  low  scores  on  MCEs.  MCE 
scores  appeared  not  to  be  related  to  scores  in  other  areas.  How- 
ever, MCE  performance  seemed  to  be  a  significant  factor  in  discri- 
minating between  the  high  and  low  scoring  PSROs.  A  discussion  of 
the  MCE  process   at  the  time  of  the  site  assessments  follows. 

MCEs  are  performed  at  the  hospital  level,  especially  in  areas 
where  a  review  program  existed  prior  to  PSRO.  Areawide  MCE  activ- 
ity, however,  has  taken  place  on  a  limited  basis  primarily  because 
such  activity  requires  extensive  aggregation  of  disparate  data 
points  and  the  coordination  of  large  numbers  of  people.  The 
site  reports  indicated  that  feedback  provided  to  hospitals  following 
monitoring  of  their  MCEs  is  very  uneven,  as  is  the  use  of  MCE  data. 
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TABLE  73 

CONCURRENT  REVIEW:     STRENGTHS  AND  WEAKNESSES 


Number  of  Times 
Identified  As  A 


Strength  Weakness 

1 •  Coordinators 

Training  3  2 

Role  2  1 

Use  of  criteria  0  5 

Competency  5  0 

2*     Physician  Advisor 

Understanding  of  role  1  5 

Qualifications  1  3 

Objectivity  0  2 

Use  for  non-delegated  review  0  3 

Training  0  3 

Use  0  5 

3.     Review  System  and  Components 

Review  features  5  0 

Refinement  of  review  systems  5  4 

Feedback  to  hospitals  3  1 

Discharge  planning  2  1 

Monitoring  7  5 
Implementation  and  accomplishment 

of  objectives  4  3 
Communication  and  relationships 

with  hospitals  7  4 

Delegated  review  2  2 

Uniform  procedures  1  2 

Coordination  of  activities  0  2 

Documentation  4  5 

Focused  review  4  5 

Physician/hospital  support  4  0 

Committee  1  3 

Technical  assistance  4  0 
Review  Coordinator/Physician 

Advisor  relations  2  2 

Unit  costs  0  2 
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TABLE  73  (Continued) 
CONCURRENT  REVIEW:     STRENGTHS  AND  WEAKNESSES 

Number  of  Times 
Identified  As  A 


Strength  Weakness 

4.  Concurrent  Review  Staff 

and  Operations 

Review  management/organization  5  1 

Training  program  and  materials  2  5 

Staff  9  1 

5.  Specific  Review  Procedures 

Physician  Advisor  referral  rate  1  5 

Efficient  review  of  denials  2  0 

6.  Profiling  and  Data  Analysis 

Level  of  analysis  and  techniques  used        3  1 

Expertise  1  2 

Frequency  and  dissemination  of  reports      1  2 

Planning  3  4 

Use  of  profile  analysis  in  review  0  2 

Understanding  value  of  3  0 

7.  Norms  and  Criteria 

Development  of  3  3 

Committee  7  3 

Periodic  review  and  revision  6  4 

Uniform  norms  1  3 

Analysis  of  problems  1  1 

Validation  of  criteria  2  1 

Validation  of  norms  1  2 

Uniform  criteria  0  3 

Dissemination  of  criteria  1  2 

Adoption  of  criteria  2  0 

Diagnoses  0  2 

Level  of  care  1  2 

Use  of  1  2 

8.  Delegation  of  Review 

Implementation  4  2 

Monitoring  system  4  3 

Delegated  hospital  relations  2  1 

Delegation  criteria  2  1 
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Summaries  applicable  to  these  items  are  shown  below: 


Quality  of  Feedback  Provided  to  Hospitals 
Following  Monitoring  of  Their  MCEs 


Constructive  criticisms  offered  (n=8) 

Pro  forma  monitoring  only  (n=3) 

Feedback  very  limited  (n=5) 

Not  reported  (n=2) 

Extent  to  Which  MCEs  are  Utilized 

Greatly  utilized  (n=2) 

Somewhat  utilized  (n=4) 

Very  little  utilization  (n=4) 

No  utilization  (n=3) 

Not  reported  (n=5) 


The  overall  quality  of  MCEs  was  judged  as  good  in  half  of 
the  PSROs  visited  and  poor  in  the  other  half.  All  sixteen  site 
reports  which  evaluated  MCE  activity  indicated  that  deficiences  of 
some  kind  existed  in  hospital  MCEs.  Some  of  the  more  frequently 
cited  criticisms  included: 


Lack  of  hospital -level  MCE  staff  training 

(n=2) 

Limited  use  of  health  care  professionals 
other  than  physicians 

(n=5) 

No  formal  method  of  topic  selection 

(n=7) 

Inaccurate  reporting  of  hospital  data 

(n=4) 

Lack  of  Concurrent  Review-MCE  coordination 

(n=2) 

Absence  of  MCE  followups 

(n=11) 

Strengths  and  weaknesses  of  MCEs  enumerated  in  the  reports  are 
listed  in  Table  74. 

F.    Other  Review 

On  the  coordinator  ratings,  the  "other  review"  category  con- 
tained three  questions  on  ancillary  services  reviewed,  physician 
services  reviewed,  and  the  extent  that  health  care  paractitioners 
other  than  physicians  are  participating  in  review.  Scores  in  this 
area  did  not  appear  to  be  related  to  other  components.  As  might  be 
expected,    older  PSROs   achieved  higher   ratings   on   this  component. 
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TABLE  74 

MEDICAL  CARE  EVALUATIONS:       STRENGTHS  AND  WEAKNESSES 

Number  of  Times 
Identified  As  A 
Strength  Weakness 


Evaluation  and  feedback 

5 

3 

Board  and  committee  involvement 

7 

2 

Hospital  cooperation 

6 

1 

Staff 

3 

3 

Topic  selection 

0 

7 

Re-studies 

0 

4 

Areawide  studies 

3 

6 

ReDortino 

1 

2 

Use  of  data 

1 

2 

Quality 

1 

3 

Staff  roles 

1 

1 

Training 

2 

3 

Physician  interest 

1 

3 

Technical  assistance  to  hospitals 

1 

2 

Monitoring 

2 

4 

Data  validity 

0 

3 

Average  spent  per  study 

0 

2 

Impact 

5 

3 

Criteria 

3 

1 

G.  Denials/Sanctions/Fraud  and  Abuse 

There  were  four  questions  in  this  category  on  the  coordinator 
rating  instrument  including  those  on  PSRO  involvement  in  denials  and 
sanctions  as  well  as  notification  to  the  State  or  DREW  of  fraud  or 
abuse.  This  is  an  unusual  category  in  that  a  case  can  be  made  for 
high  scores  being  indicative  of  either  good  or  bad  performance* 
However,  those  projects  designated  as  "best"  in  other  components 
tended  to  achieve  high  marks  in  this  area  as  well* 

H.  Responsiveness/Innovativeness  of  Review 

Seven  questions  on  the  coordinator  rating  instrument  were  used 
to  measure  this  area.  This  is  a  broad  category  covering  such  areas 
as  use  of  data  to  identify  utilization  problems/  development  of 
action  plans  to  address  problems  and  other  innovations  in  review* 
Scores  on  this  component  appeared  to  be  highly  related  to  organi- 
zational strength  and  general  management.  Older  PSROs  did  particu- 
larly well  in  this  area.  The  category  was  also  found  to  be  a 
useful  set  of  items  in  discriminating  between  low  scoring  and  hig^ 
scoring  PSROs. 
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I.     Impact  and  Formal  Monitoring 


In  addition  to  the  eight  explicit  components  just  discussed,  a 
problem  frequently  cited  in  the  assessment  reports  was  the  lack  of 
formal  monitoring  by  PSROs  of  the  cost  and  impact  of  review. 
The  site  reports  almost  universally  assailed  the  absence  of  a 
"formal  mechanism"  to  monitor  both  expenditures  and  review  services 
impacts . 

Ten  site  reports  recommended  that  PSROs  implement  hospital 
monitoring  systems  complete  with  a  formal  mechanism  for  dissemi- 
nating monitored  information  once  gathered. 

Little  information  is  provided  in  the  reports  from  which 
to  directly  assess  the  cost-effectiveness  of  the  various  PSRO 
functions.  The  assessors  were  careful  to  point  out  that  the  PSROs 
themselves  could  not  make  these  judgments  due  to  the  absence  of 
continuous  and  systematic  monitoring  systems.  Indeed,  the  major 
criticism  leveled  by  the  site  teams  is  that  the  PSROs  are  presently 
unable  to  evaluate  the  results  of  their  own  efforts. 

Notwithstanding  these  limitations,  positive  impacts  were  recog- 


a 

number  of  important  areas: 

• 

Increased  case  mix  severity 

(n=1) 

■ 

Improved  quality  of  physician  documentation 

(n=7) 

• 

Improved  quality  of  MCEs 

(n=2) 

• 

Reduced  number  of  Medicare  admissions 

(n=3) 

■ 

Reduced  number  of  Medicaid  admissions 

(n=3) 

• 

Reduced  Medicare  ALOS 

(n=8) 

• 

Reduced  Medicaid  ALOS 

(n=8)^1 

Many  of  these  impacts  could  not  be  attributed  to  specific 
actions  taken  by  the  PSROs.  For  example,  while  ten  PSROs  did  not 
have  a  formal  mechanism  to  identify  and  document  discharge  planning 
impacts,  nine  were  noted  by  the  site  teams  to  have  expressed  a 
noticeable  improvement.  It  is  assumed  that  this  improvement  is  due 
to  the  efforts  of  the  PSROs. 

6.2.4.3    Comparative  Analyses 

Based  on  the  foregoing  assessments,  overall  performance  of  the 
PSROs  was  analyzed  and  related  to  variables  thought  to  have  a 
potential  effect  upon  performance.  The  results  of  these  analyses 
are  discussed  below. 


/ 1    One  site  report  noted  an  increase  in  Medicaid  ALOS. 
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A*     "High/Low"  Comparisons 


After  coding  each  of  the  site  reports,  abstractors  doing 
the  content  analysis  rated  how  well  they  thought  each  PSRO  was 
managed.  Among  the  eighteen  PSROs ,  four  were  deemed  "very  well 
managed,"  seven  were  judged  as  "fair,"  and  seven  were  classified  as 
"poorly  managed".  An  analysis  of  the  various  characteristics 
evaluated  in  the  original  site  reports  was  then  performed  with 
request  to  the  two  groups:  "very  well  managed"  and  "not  well 
managed".     The  comparative  results  are  displayed  in  Table  75. 

Attempts  were  then  made  to  ascertain  whether  the  well-managed 
PSROs  indeed  outperformed  the  not-well-managed  by  use  of  independent 
measures  not  provided  by  the  site  teams.  The  independent  measures 
selected  were:  levels  of  MCE  and  other  review  activities,  referrals 
to  physician  advisors,  and  denials  for  both  admission  certification 
and  continued  stay  review.  These  data  were  abstracted  from  a  number 
of  tapes  and  reports.  In  considering  the  results,  however,  a  caveat 
is  in  order:  the  findings,  presented  in  Tables  76  and  77,  are 
very  preliminary.  Although  some  of  the  findings  may  be  suggestive, 
the  number  of  cases  upon  which  the  analysis  is  based  is  not  very 
large  nor  were  the  cases  randomly  selected. 

In  Table  76,  the  "well-managed"  and  "not-well-managed"  PSROs 
are  compared  in  terms  of  the  percentage  of  cases  referred  by 
CR  coordinators  to  physician  advisors  and  the  percentage  denied 
further  health  insurance  benefits  in  the  second  quarter  of  1977. 
Each  of  these  parameters  represents  the  most  recent  data  available. 
The  differences  were  generally  not  statistically  significant,  the 
only  exception  being  in  the  area  of  admission  certifications  denied, 
which  tended  to  be  higher  among  the  well-managed  PSROs. 

Further  investigation  showed  an  absolute  decline  in  both 
days  of  care  per  1,000  Medicare  eligibles  and  in  average  length  of 
stay  for  both  the  "well-managed"  and  "not-well-managed"  PSRO  groups, 
with  the  average  declines  slightly  greater  among  the  latter  group. 
The  differences  between  the  groups,  however,  were  too  small  to  be 
statistically  significant. 

As  shown  in  Table  77,  medical  care  evaluation  activity  measures 
showed  a  higher  average  number  of  audits  per  hospital,  a  higher 
average  number  of  total  hours  (physician  and  non-physician)  per 
audit,  and  a  higher  average  number  of  physician  hours  per  audit 
in  the  "not-well-managed"  group  as  compared  to  the  "well-managed" 
group.  Because  of  the  small  number  of  areas  compared,  however,  the 
differences  were  not  statistically  significant  for  two  of  the  three 
activity  measures  analyzed;  the  one  exception  was  the  average  number 
of  total  hours  per  audit. 
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TABLE  75 

ASSESSORS'    SUMMATIVE  CATEGORIZATION  OF  PSRO  MANAGEMENT  FOR  USE 
IN  COMPARATIVE  WELL-MANAGED/NOT-WELL-MANAGED  ANALYSIS: 
EMPIRICAL  CONFIRMATION  BY  DISTRIBUTION  OF  INDIVIDUAL  TOPIC 

FINDINGS 


Characteristic 

Assessors  Judgement: 
How  Well  PSRO  Is  Managed 

Very  Well 
(n=4) 

Not  Well 
(n=7) 

Committee  Effectiveness 

Outstanding  ( 2 ) 

Outstanding  (  1 ) 
Good  (2) 

IPs  4  v     I  O  \ 

r  air  \ z ) 
Poor  ( 1 ) 

yuaiity  ot  v^oininunicauions 
Between  Board  and  Committee 

Good  (  J} 

Good  1 1 ; 
Poor  (3) 

Between  Committee  and  Staff 

Good  \Z) 

Good  (  1  ) 
Poor  (3) 

Division  of  Responsibility 

Good  (2) 

Unsatisfactory  (4) 

yuanty  or  formal  Training 
Given  Staff 

Good  {  1 ) 

Good  ( 1 ) 
Fair  ( 1 ) 

No  training  given 

Existence  of  Formal  Mechan- 
isms to  Identify  Own  Impact 

Yes  (2) 
No  (1) 

Yes  ( 1 ) 
No  (5) 

Degree  to  Which  PSRO  Util- 
izes Own  Monitoring  Results 

Adequately  ( 3 ) 

Minimally  (4) 
Not  at  all  ( 1 ) 

Adequacy  of  PSRO  Monitor- 
ing of  Hospital  CR 

Very  adequately  ( 1 ) 
Adequately  ( 1 ) 

Adequately  ( 2 ) 
Inadequately  ( 4 ) 

Degree  to  which  MCEs  are 
Utilized 

Greatly 

utilized  (2) 
Somewhat 

utilized  (  1 ) 

Somewhat  utilized 
Very  little 

utilization  ( 1 ) 
No  utilization  (4) 

Quality  of  MCEs 

Good  (3) 
Adequate  ( 1 ) 

Adequate  ( 1 ) 
Poor  (6) 

How  Actively  PSRO  Works  to 
Rectify  Management  Problems 

Very  actively  (2) 
Actively  (2) 

Actively  ( 1 ) 
Little  activity  (2 
Does  not  address 
problems  ( 3 ) 

Utilization  of  PAs  in 
Review  Programs 

Somewhat  effective 
(2) 

Somewhat  effective 
Not  very  effective 
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PSROs  which  had  consistently  high  scores  on  the  components  in 
the  coordinator  ratings  and  those  which  had  consistently  low  scores 
were  selected  for  further  analysis.  From  this  set  of  fourteen, 
those  which  matched  the  "best"  and  "worst"  in  the  analysis  described 
above  were  then  selected  for  in-depth  analysis.  There  were  three  in 
the  "high"  category  and  four  in  the  "low".  Average  scores  for  the 
high  and  low  groups  are  displayed  in  Table  78. 

TABLE  78 

Average  Scores  on  Management  Components  for  PSRO 
Performance  Groupings 


Overall 

Performance  Level 

LOW  (4  PSROs) 

HIGH  (3  PSROs) 

Category 

Average  Score 

Average  Score 

Organizational  Strength 

11 

17 

General  Management 

14 

26 

External  Relationships 

9 

10 

Concurrent  Review 

8 

14 

MCEs 

1 

6 

Other  Review 

1 

6 

Denials/Sanctions/Fraud/Abuse 

0 

3 

Responsiveness  of  Review 

3 

17 

Total,  All  categories  combined 

46 

96 

B.    Age  Comparisons 


Scores  on  the  site  assessment  coordinator  rating  scales  were 
analyzed  to  determine  whether  the  PSRO  performance  ratings  were 
a  function  of  their  age.  In  this  analysis,  age  was  defined  as  the 
time  between  conditional  status  designation  to  the  time  of  site 
assessment.  Given  the  small  number  of  cases  available,  PSROs  were 
divided  into  high,  medium,  and  low  categories  for  both  age  and  for 
their  scores  on  each  of  the  rating  components.  The  assignment  of  a 
PSRO  to  a  high,  medium  or  low  category  was  based  on  its  position  in 
the  distribution  for  the  item.  The  26  PSROs  examined  in  this 
analysis  ranged  in  age  from  17  months  to  5  0  months.  Those  placed  in 
the  "low"  age  category  (9  PSROs)  were  less  than  2  years  old;  those 
placed  in  the  "medium"  category  (8  PSROs)  were  29-39  months  old; 
those  placed  in  the  "high"  category  (9  PSROs)  were  42  months  or 
older. 

The  number  of  PSROs  falling  into  the  respective  categories  for 
both  age  and  each  of  the  eight  measures  are  presented  in  Table  79. 
Although  by  no  means  conclusive,  the  data  suggest  that  older  PSROs 
tend  to  rate  higher  with  respect  to  MCEs,  external  relations,  and 
responsiveness  of  review.  None  of  the  relationships  is  statistic- 
ally significant,    however,    because  of   the  small  numbers  involved. 


258 


TABLE  79 

RATINGS  ON  QUESTIONNAIRE  COMPONENTS  BY  AGE  OF  PSRO 

MCES 


Organizational  Strength 


H 

M 

L 

H 

M 

L 

H  3 

4 

2 

Age    H  4 

o 
e. 

3 

M  2 

3 

3 

M  2 

4 

2 

L  3 

2 

4 

L  1 

4 

4 

General  Management 

Other  Review 

H 

M 

L 

H 

M 

L 

H  3 

3 

3 

Age      H  6 

3 

M  4 

2 

2 

M  - 

8 

L  2 

3 

4 

L  2 

4 

3 

External  Relations 

Denials/Sanctions 

H 

M 

L 

H 

M 

L 

H  4 

4 

1 

Age      H  3 

2 

4 

M  2 

3 

2 

M  2 

2 

4 

L  2 

5 

3 

L  1 

5 

3 

Concurrent 

Review 

Responsiveness  of 

Review 

H 

M 

L 

H 

M 

L 

H  5 

2 

2 

Age      H  4 

4 

1 

M  2 

3 

3 

M  3 

3 

2 

L  3 

4 

2 

L  2 

2 

5 

Age  H 


Total 

H 

M 

L 

H 

4 

2 

3 

M 

3 

2 

3 

L 

2 

3 

4 

1/    Age  is  computed  as  ti-e  f«-  conditional  status  designation  to 
time  of  site  assessment. 
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It  can  be  argued  that  the  PSROs  with  the  most  extensive  experi- 
ence were  those  originally  funded  by  the  National  Center  for  Health 
Services  Research  as  Experimental  Medical  Care  Review  Organizations 
(EMCROs)  prior  to  the  passage  of  the  PSRO  legislation.  There  were 
five  of  these  projects  among  the  site  assessment  group.  The  scores 
of  the  EMCROs  were  compared  with  those  scored  as  high  performers  in 
the  content  analysis,  and  with  the  balance  of  the  PSROs  partic- 
ipating in  the  site  assessments.  The  results,  displayed  in  Table 
80,  show  that  the  former  EMCROs  and  the  "high"  performance  PSROs 
have  essentially  the  same  scores  on  all  items  except  MCEs,  in  which 
the  former  EMCROs  are  markedly  higher.  This  may  reflect  the  orient- 
ation of  the  EMCRO  program  toward  quality  assessment  and  quality 
assurance. 

TABLE  80 

COMPARISON  OF  RATINGS  ON  EIGHT  PERFORMANCE  CATEGORIES 


Performance 
Categories 
(Range  of  Possible 
(Scores) ) 

Content  Analysis 
High  Performers 

(N=3) 

Former 
EMCROs 

(N=5)  • 

Others 
(N=18) 

Organizational 

Strength 

(0-28) 

17 

16 

14 

General 
Management 
( 0-35 ) 

26 

24 

18 

External 

Relations 

(0-16) 

10 

10 

9 

Concurrent 

Review 

(0-23) 

14 

14 

10 

MCEs 
(0-20) 

10 

17 

3 

Other 

Review 

(0-10) 

3 

5 

2 

Denials/ 

Sanctions 

(0-16) 

3 

1 

1 

Responsiveness 
to  Review 
(0-27) 

17 

19 

19 
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C.    Regional  Comparisons 


Since  other  evidence  has  suggested  that  PSROs  vary  with 
regional  location,  a  regional  analysis  was  performed.  PSROs  were 
divided  into  three  categories  according  to  census  region:  West  (9 
PSROs),  Northeast  (10  PSROs),  North  Central  (5  PSROs),  and  South  (2 
PSROs).  Average  scores  for  the  eight  components  are  displayed  in 
Table  81. 


TABLE  81 

COMPARISON  OF  RATINGS  ON  EIGHT  PERFORMANCE  CATEGORIES: 
Average  Scores  by  Region  of  the  Country  -  26  PSROs 


North 

North- 

West 

Central 

South 

east 

(9) 

(5) 

(2) 

(10) 

Organizational  Strength 

15 

15 

16 

15 

General  Management 

21 

17 

27 

19 

External  Relations 

10 

9 

10 

10 

Concurrent  Review 

11 

11 

16 

11 

MCEs 

6 

1 

9 

4 

Other  Review 

3 

1 

6 

2 

Sanctions/Denials 

1 

0 

4 

1 

Responsiveness  of  Review 

12 

5 

18 

15 

Average  scores,  by  component,  show  no  striking  dissimilar- 
ities among  regions,  with  the  exception  of  the  MCE  and  Responsive- 
ness of  Review  categories.  However,  when  the  distributions  of 
scores  are  examined,  several  differences  are  apparent.  North 
Central  PSROs  tended  to  have  few  high  scores  in  any  of  the  com- 
ponents, ranking  lowest  (or  tied  for  lowest)  in  all  eight  cate- 
gories. The  failure  of  North  Central  PSROs  to  achieve  high  scores 
may  be  a  function  of  several  factors  including:  their  relatively 
small  number  in  this  analysis,  their  relatively  young  age,  and  the 
fact  that  there  are  no  former  EMCROs  among  the  North  Central  PSROs 
included  in  the  analysis. 

6.2.5     PSRO  Implementation  of  Site  Assessment  Recommendations 

Site  assessments  are  used  by  HSQB  as  a  management  tool  to 
identify  strengths  and  problems  in  PSROs,  and  to  recommend  solutions 
for  them.  The  assessment  reports  contain  specific  recommendations 
for  improving  a  PSRO's  efficiency  and  effectiveness.  Some  of  the 
identified  problems  are  resolved  soon  after  the  visit;  others  of  a 
more  difficult  nature  are  addressed  over  time  by  the  PSRO  and  the 
project  officer. 
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HSQB  had  addressed  problem  areas  identified  in  assessments 
through  technical  assistance  and  policy  revision.  Examples  of 
actions  taken  include: 

1 .  Management  of  the  PSRO  program  -  It  was  observed  during  assess- 
ments that  PSROs  were  generally  good  performers  in  accomp- 
lishing "compliance"  activities,  such  as  contract  deliverables, 
but  tended  to  consider  compliance  with  DHEW  policy  as  the  index 
by  which  their  performance  was  measured,  rather  than  as  one  of 
several  possible  means  to  achieve  the  larger  goal  of  impact  on 
the  health  care  delivery  system.  Emphasis  was  on  completion  of 
specific  tasks  rather  than  on  setting  objectives  and  trying  to 
achieve  them.  To  change  this,  HSQB  has  instituted  a  program  of 
objective-setting  for  each  PSRO.  (See  Section  7.3.3  for  a 
description  of  this  effort. ) 

2.  Profile  analysis  -  Through  evaluating  problems  in  initiating 
profile  analysis,  it  was  apparent  that  some  PSROs  lacked  the 
sophistication  needed  to  realize  the  full  potential  of  their 
data  systems,  for  example,  through  the  development  and  testing 
of  hypotheses  about  the  cause  of  excessive  lengths  of  stay.  A 
technical  assistance  effort  is  underway  to  help  PSROs  to  better 
utilize  their  data  systems  for  profile  analysis  and  impact 
studies. 

3.  Medical  care  evaluation  (MCE)  studies  -  The  low  proficiency 
level  of  some  PSROs  and  the  failure  to  meet  numerical  require- 
ments by  others  has  prompted  closer  monitoring  of  MCE  perform- 
ance and  the  establishment  of  a  national  MCE  Clearinghouse,  a 
technical  assistance  hotline,  and  recent  publication  of  a 
handbook  for  the  conduct  of  MCEs. 

4.  Monitoring  of  review  -  Problems  identified  in  the  monitoring 
procedures  used  by  some  PSROs  have  resulted  in  increased 
attention  and  technical  assistance  in  this  area.  PSROs  are 
being  held  increasingly  more  accountable  for  the  cost  and 
quality  of  review  in  delegated  facilities. 

5.  Unit  costs  of  review  -  Progress  in  the  determination  of  the 
unit  cost  of  concurrent  review  has  been  spurred  by  the  desire 
to  develop  information  on  the  efficiencies  of  review  for 
comparative  purposes.  Increased  attention  to  modifying  the 
concurrent  review  system,  through  reducing  the  intensity  of 
review  where  it  is  not  cost-effective  and  intensifying  review 
where  it  is  felt  it  can  be  more  effective  (focusing)  is  one  of 
HSQB's  primary  efforts  to  reduce  the  unit  cost  of  review.  The 
need  to  control  costs  of  delegated  review  has  resulted  in 
better  communication  between  PSROs  and  Medicare  fiscal  inter- 
mediaries. 
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6.  Impact  analysis  -  Analysis  of  the  impact  of  PSROs  has  been 
hindered  by  the  lack  of  full  implementation  of  the  program. 
Assessments  have  prompted  implementation  of  all  systems  within 
PSROs — review,  data,  and  monitoring — and  has  increased  the 
awareness  of  PSROs  that  they  are  accountable  for  demonstrating 
impact* 

7.  Self -assessment  -  The  HSQB  assessment  process  has  introduced 
methods  by  which  PSROs  may  conduct  self -assessments .  Several 
aspects  of  the  process,  including  the  use  of  peer  teams  to 
promote  cross-fertilization  of  ideas  and  the  dissemination  of 
assessment  materials  throughout  the  program,  are  encouraging 
PSROs  to  internally  evaluate  all  aspects  of  their  organi- 
zations. 

8.  External  relations  -  HSQB  has  recognized  that  in  some  cases 
external  organizations  were  hampering  the  progress  of  PSROs. 
Problems  with  State  Medicaid  agencies  have  been  alleviated  to 
some  degree  by  involvement  of  agency  representatives  as 
observers  on  assessment  teams. 

9.  Data  quality  control  -  Identification  of  problems  in  this  area 
has  resulted  in  the  publication  of  a  technical  assistance 
document  on  data  quality  control  and  the  award  of  a  contract  to 
provide  technical  assistance  to  PSROs  in  this  area. 

10.  Greater  program  awareness  and  appreciation  of  the  byproducts  of 
PSRO  review  -  Assessments  have  resulted  in  identification  of 
certain  beneficial  side  effects  of  review,  including: 

-  improved  quality  of  medical  record  and  physician  documenta- 
tion 

-  use  of  "waiting  days"  data  to  evaluate  the  availability  of 
nursing  home  beds 

improved  quality  of  hospital  discharge  data. 

A  study  conducted  by  HSQB  staff  indicates  that  the  assess- 
ment reports  are  of  utility  to  the  PSROs.  As  of  August  15,  1978, 
24  PSROs  had  been  given  an  opportunity  to  provide  HSQB  with  comments 
on  the  draft  assessment  report  prior  to  its  issuance  as  a  public 
document.  The  PSROs  were  asked  to  include  in  their  responses  any 
factual  errors  in  the  reports,  and  misinterpretations  by  assessment 
team  members.  In  addition,  many  PSROs  provided  updates  on  self- 
initiated  actions  taken  on  the  team's  recommendations.  Eleven  of 
these  unsolicited  updates  were  randomly  selected  and  the  information 
contained  was  analyzed.     The  subject  areas  shown  in  Table  82  were 
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TABLE  82 

NUMBER  OF  SITE  ASSESSMENT  RECOMMENDATIONS 
IMPLEMENTED  WITHIN  THREE  MONTHS 


Subject  Areas 


No.  of  Assessment 
Reports  Which  Cited 
Recommendation  in 
This  Area 


No.  of  PSROs 
Which  Imple- 
mented Recom- 
mendations 
Within  3  Months 


Program  Management 
Set  objectives 

Increase  physician  involvement 
Increase  Board  involvement  in 

PSROs  financial  status 
Management  staff  changes 
Committee  structure  changes 
Begin  staff  training  activities 

Quality  Assurance  Activities 

Revise  and  implement  monitoring 

plan 
Focus  review 

Criteria  revisions  and  updates 
recommended 


7 
4 


n 
n 


Data  Management 

Hire  Data  Manager 
Change  Data  Processor 


Professional  and  External  Relations 


Improve  relationship  with  area 

hospitals 
Improve  relationship  with  HSA 
Improve  relationship  with  area 

Medical  Society 

Totals 


6 
6 

3 

69 


45  (65%) 
42  (61%)  /3 


/1  One  action  based  on  PSRO  analysis  rather  than  specific  team 
recommendation . 

/2  Two   actions    based   on   PSRO   analysis   rather  than   specific  team 

recommendation. 

/3  Not  including  actions  cited  in  footnotes  /1_  and  /2. 
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selected  for  the  analysis  since  they  represent  current  HSQB  prior- 
ities. The  eleven  assessment  reports  were  then  culled  for  recom- 
mendations fitting  those  priority  areas.  The  number  of  PSROs 
addressing  recommendations  in  these  areas  was  compiled  from  the 
information  available  and  is  displayed  in  Table  82.  Over  sixty 
percent  of  the  assessment  recommendations  in  priority  areas  are  seen 
to  have  been  implemented  within  three  months  of  feedback. 

6.3    Hospital  Review  Activity  Analysis 

6.3.1  Background 

The  PSRO  Management  Information  System  (PMIS)  files  contain 
data  on  utilizaton  review  process  measures/  such  as  the  number  of 
admission  reviews  performed,  the  number  of  continued  stay  reviews 
performed,  the  number  of  cases  referred  to  physician  advisors,  and 
the  number  of  denials  for  each  type  of  review.  These  data  were 
studied  to  determine  general  levels  of  activity  for  a  variety  of 
cross-sectional  operational  measures;  and  to  explore  whether  there 
was  an  increased  scrutiny  of  cases  and  denials  involving  diagnostic 
conditions  which  are  considered  more  likely  to  present  problems  with 
inappropriate  utilization. 

Because  of  the  considerable  differences  in  volume  of  activity 
among  PSROs,  an  attempt  was  made  to  express  each  of  the  parameters 
with  respect  to  an  appropriate  denominator.  For  the  continued  stay 
review  measures,  the  denominator  was  the  number  of  covered  cases  for 
which  a  continued  stay  review  was  performed.  Unfortunately,  the 
data  base  does  not  contain  a  comparable  measure  for  admission 
review.  There  is  no  precise  measure  of  the  number  of  patients  for 
whom  admission  certification  review  was  conducted.  The  number  of 
hospital  discharges  potentially  subject  to  admission  review  and  the 
number  of  admission  certifications  actually  granted  exist  but  the 
average  difference  between  these  two  numbers  is  considerably  greater 
than  the  number  of  reported  denials. /I 

For  these  reasons,  ratio  measures  which  express  the  number'  of 
referrals  and  number  of  denials  relative  to  the  total  number  of 
admission  certifications  granted  were  used  to  characterize  the 
admission  review  process.  While  this  measure  differs  from  the 
percentage  measure  of  the  continued  stay  review  process,  it  suffices 
to  provide  useful  insight. 


/1  Reporting  inconsistencies  exist  between  sources  of  data.  The 
121  form  does  not  include  exempted  discharges  and  discharges 
where  only  continued  stay  review  is  performed. 
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6*3.2  National  and  Regional  Levels  of  Case  Referrals 
to  Physician  Advisors  and  Denials  of  Continued 
Health  Insurance  Coverage 

Table  83  summarizes  the  level  of  concurrent  review  activity 
in  terms  of  cases  reviewed,  certifications  granted,  referrals  to 
physician  advisors,  denials,  etc.,  during  the  second  quarter 
of  1977  (April  1  to  June  30,  1977)  for  the  42  PSROs  in  the  aggregate 
PSRO  Hospital  Discharge  Data  Set  (PHDDS)  file.  Since  the  distri- 
butions are  all  skewed  to  the  right,  both  means  (with  standard 
deviations)  and  medians  as  presented. 

TABLE  83 
PSRO  REVIEW  PROCESS  MEASURES 
2nd  Quarter,  1977 


Variable 

Mean 

+ 

S.D. 

Median 

Total  Medicare  and 

Medicaid  Discharges 

14,019 

+ 

11,899 

9,440 

Medicare 

9,808 

+ 

8,417 

6,979 

Medicaid 

4,621 

+ 

3,963 

3,002 

Admission  Certifications 

13,420 

+ 

11,524 

8,741 

Granted 

Referred  to  Physician 

768 

+ 

1,000 

336 

Advisor 

Denials 

13 

+ 

21 

2 

Continued  Stay  Reviews 

6,914 

+ 

5,371 

4,988 

Performed 

Referred  to  Physician 

2,547 

+ 

2,533 

1,471 

Advisor 

Denied 

138 

+ 

123 

85 

On  a  national  basis,  there  were  approximately  57  admission 
reviews  referred  to  physician  advisors  for  every  1,000  certifi- 
cations granted  (i.e.,  768  -  13.42),  but  only  one  admission  certifi- 
cation denied  for  every  1,000  granted.  In  contrast,  368  of  every 
1,000  continued  stay  reviews  were  referred  to  physician  advisors  and 
almost  19  of  every  1,000  reviews  resulted  in  a  denial  of  continued 
health  insurance  benefits. 
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The  denial  rates  reported  by  these  42  PSROs  are  reasonably 
comparable  to  the  levels  found  in  a  19  76-77  national  survey  of 
hospital  UR  programs. /!_  In  the  cited  study,  the  denial  rates  were 
one  per  thousand  admission  reviews  (no  difference)  and  11  per 
thousand  continued  stay  reviews  (as  opposed  to  19). 

The  interpretation  of  continued  stay  review  referrals  to 
physician  advisors  is  difficult  since  some  PSROs  have  rules  which 
automatically  require  a  case  to  be  referred  if  it  is  to  be  certified 
past  a  pre-determined  percentile  of  the  average  length  of  stay  for 
cases  in  that  area  with  similar  diagnoses  in  the  same  age  group.  In 
some  PSROs,  this  rule  is  used  for  all  cases  being  certified  at  the 
75th  percentile  level;  thus,  for  these  PSROs,  one  would  expect  a 
substantial  percentage  of  reported  referrals  to  be  based  on  such 
automatic  rules,  whereas  for  other  PSROs  referrals  involve  an 
explicit  expression  of  concern  by  utilization  review  coordinators 
that  the  case  does  not  warrant  continuance  at  an  acute  hospital 
level  of  care.  Some  PSROs  also  apply  automatic  referral  procedures 
to  admission  review. 

Table  84  shows  the  distribution  of  admission  and  continued  stay 
review  referrals  and  denials  for  the  four  major  census  regions.  The 
level  of  activity  is  far  higher  in  the  Northeast  than  in  the  rest  of 
the  country  for  all  except  the  percentage  of  continued  stay  reviews 
referred  to  physician  advisors.  However,  the  mean  difference  among 
regions  approaches  statistical  significance  only  for  the  admission 
denials  measure  (p  =  0.068). 

For  the  42  PSROs,  correlation  coefficients  were  calculated 
for  the  following  pairs  of  variables:  (a)  admission  certifications 
vs.  continued  stay  reviews,  (b)  admission  referrals  vs.  continued 
stay  referrals,  and  (c)  admission  certification  denials  vs.  con- 
tinued stay  denials.  Each  pair  was  found  to  be  highly  intercorre- 
lated  with  product-moment  correlations  of  0.97,  0.78  and  0.61 
respectively;  each  of  these  correlation  values  is  significantly 
different  from  zero  at  the  0.001  level. 

Further  correlation  computations  showed  the  volume  of  continued 
stay  referrals  and  continued  stay  denials  to  be  moderately  cor- 
related (r  =  0.57,  p  =  0.001)  whereas  admission  certification 
referrals  and  denial?  were  found  to  have  a  product-moment  score  of 
only  0.16  (p  =  0.15}.  However,  after  adjustments  were  made  to 
denials  and  referrals-,  for  volume  of  review,  all  correlations  (in- 
cluding admission  referrals  vs.  denials)  were  positive  and  signifi- 
cant. 


/1     See  Gertman,   et.  al.,  Utilization  Review  in  the  U.S.;  Results 
from  a  1976  National  Survey  of  Hospitals. 
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This  interrelationship  of  admission  review  measures  and 
continued  stay  review  measures  is  a  probable  indication  of  syste- 
matic behavior  in  review  processes  by  PSROs. 

The  Relationship  of  Diagnosis  to  Levels  of  Denials  and  Refer- 
rals to  Physician  Advisors 

The  PHDDS  File  reports  a  record  for  each  of  twenty-three 
selected  diagnosis  groups  with  respect  to  the  number  of  discharges, 
number  of  admission  certifications,  admission  denials,  etc . /I 
Based  on  experience  with  hospital  UR  and  PSRO  programs,  it  was 
believed  possible  to  identify  a  priori  diagnoses  which  had  a  higher 
frequency  of  inappropriate  admissions  and  inappropriate  prolongation 
of  hospitalization.  For  the  23  diagnoses,  two  subgroups  were 
identified:  one  considered  likely  to  have  a  relatively  high  fre- 
quency of  admissions  referred  to  physician  advisors  and/or  denied, 
and  a  second  for  which  it  was  felt  that  admission  referrals  and 
denials  would  occur  at  a  much  lower  rate.  A  similar  dichotomy  of 
the  23  diagnoses  was  performed  with  respect  to  continued  stay 
review.  The  assignment  of  diagnoses  to  high  and  low  frequency 
groupings  for  admission  and  continued  stay  review  respectively  is 
shown  in  Tables  85  and  86  along  with  their  comparative  statis- 
tics. It  will  be  seen  that  some  diagnoses  (e.g.,  neuroses)  appear 
in  the  high  frequency  group  for  both  types  of  review,  others  appear 
in  both  low  frequency  groupings  (e.g.,  normal  deliveries),  while 
some  switch  from  the  high  to  low  frequency  group  or  vice  versa 
depending  on  the  type  of  review.  An  example  of  the  latter  is  the 
diagnosis  "fractures,"  assigned  to  the  low  frequency  grouping  for 
admissions  review  referrals  and  denials  but  to  the  high  frequency 
grouping  for  continued  stay  review  referrals  and  denials. 

Table  85  shows  that  admission  review  referrals  to  physician 
advisors  were  38  percent  higher  and  admission  review  denials  142 
percent  higher  in  the  hypothesized  high  frequency  diagnosis  group  as 
compared  to  the  low  frequency  diagnosis  group;  both  of  these  differ- 
ences were  statistically  significant  at  the  0.001  level.  Table  86 
shows  that  continued  stay  review  referrals  to  physician  advisors 
were  15  percent  higher  and  continued  stay  review  denials  67  percent 
higher  in  the  hypothesized  high  frequency  grouping  as  compared  to 
the  hypothesized  low  frequency  grouping;  again  these  differences 
were  highly  significant. 


/1  With  one  observation  for  each  of  two  quarters  times  23  diagno- 
sis-specific observations  times  42  PSROs,  there  were  a  maximum 
of  1,932  observations  for  each  parameter  of  interest. 
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TABLE  85 

ADMISSION  REVIEW:   RELATIONSHIP  OF  DIAGNOSIS  TO  FREQUENCY 
OF  REFERRALS  TO  PHYSICIAN  ADVISOR  AND  DENIAL 

HYPOTHESIZED  DIAGNOSES  FOR  REFERRALS  AND  DENIALS: 


HYPOTHESIZED 
HIGH 
FREQUENCY 


Neuroses;  diseases  of  the  eye;  acute  upper  respiratory 
influenza;  cystitis  and  other  urologic  diseases;  dis- 
eases of  the  fen  ale  genital  organs;  diseases  of  the 
skin  and  subcutaneous  tissues;  arthritis  symptom s ; 
nervous,  respiratory,  and  circulatory  diseases;  GI 
and  urologic  systems. 


HYPOTHESIZED 
LOW 
FREQUENCY 


Infectious  diseases;  diabetes;  acute  myocardial  in- 
farction; arrythmias;  cerebrovascular  diseases;  dis- 
eases of  the  vascular  system;  pneumonia;  bronchitis; 
other  respiratory  diseases;  abdominal  hernia;  gall 
bladder  and  bile  duct  diseases;  diseases  of  the 
prostate;  normal  delivery;  fractures. 


RATIO  OF  REFERRALS  TO  PHYSICIAN  ADVISORS  PER  ADMISSION  CERTIFICATION 

GRANTED 


NUMBER  OF  CASES 


MEAN  +  STANDARD  DEVIATION 


High 

Frequency  756 
Diagnoses 


0.0848  +  0.145 


Low 

Frequency  1 , 176 

Diagnoses 


0.0612  +  0.126 


t  value  =  3.69;  p  <  0.001 


RATIO  OF  ADMISSION  DENIALS  PER 
NUMBER  OF  CASES 

High 

Frequency  756 
Diagnoses 

Low 

Frequency  1,176 
Diagnoses 

t  value  =  3.83;  p  <  0.001 


ADMISSION  CERTIFICATION  GRANTED 
MEAN  +  STANDARD  DEVIATION 

0.0017  +  0.007 
0.0007  +  0.003 
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TABLE  86 

CONTINUED  STAY  REVIEW:   RELATIONSHIP  OF  DIAGNOSIS  TO  FREQUENCY 
OF  REFERRALS  TO  PHYSICIAN  ADVISOR  AND  DENIAL 

HYPOTHESIZED  DIAGNOSES  FOR  REFERRALS  AND  DENIALS: 


HYPOTHESIZED 
HIGH 
FREQUENCY 


Diabetes;  neuroses;  cerebrovascular  diseases;  dis- 
eases of  the  vascular  system;  pneumonia;  bronchitis; 
other  respiratory  diseases;  fractures. 


HYPOTHESIZED 
LOW 
FREQUENCY 


Infectious  diseases;  diseases  of  the  eye;  acute  myo- 
cardial infarction;  arrythmias;  acute  upper  respira- 
tory influenza;  abdominal  hernia;  gall  bladder  and  bile 
duct  diseases;  cystitis  and  other  urologic  diseases; 
diseases  of  the  female  genital  organs;  diseases  of 
the  skin  and  subcutaneous  tissues;  arthritis  symp- 
toms; nervous,  respiratory,  and  circulatory  diseases; 
GI  and  urologic  systems;  diseases  of  the  prostate; 
normal  delivery. 


RATIO  OF  REFERRALS  TO  PHYSICAN  ADVISORS  PER  CONTINUED  STAY  REVIEW 


NUMBER  OF  CASES 


MEAN  +  STANDARD  DEVIATION 


High 

Frequency  652 
Diagnoses 


0.3785  +  0.253 


Low 

Frequency  1,260 
Diagnoses 


0.3293  +  0.264 


t  value  =  3.96;  p  <  0.001 


RATIO  OF  CONTINUED  STAY  DENIALS  PER  CONTINUED  STAY  REVIEW 

NUMBER  OF  CASES  MEAN  +  STANDARD  DEVIATION 

High 

Frequency  672  0.0325  +  0.043 

Diagnoses 

Low 

Frequency  1,260  0.0192  +  0.038 

Diagnoses 

t  value  =  6.78;  p  <  0.001 
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Tables  87  to  90  present  two-way  analyses  of  variance 
which  examine  the  independent  and  joint  effects  of  the  hypothesized 
high  and  low  frequency  diagnosis  groupings  and  the  geographic 
regional  effects  on  each  of  the  four  parameters  of  interest. 
While  the  hypothesized  high  frequency  group  consistently  displays 
averages  above  that  of  the  hypothesized  low  frequency  group  in  each 
of  the  four  geographic  regions,  the  rate  of  referrals  and  denials  in 
the  low  frequency  group  in  the  Northeast  frequently  exceeds  that  of 
the  high  frequency  group  in  the  other  three  regions.  Both  the 
diagnosis  group  effect  and  the  regional  effects  are  highly  signi- 
ficant for  all  four  measures  with  p  values  below  0.001. 


TABLE  87 

DIFFERENCES  IN  ADMISSION  REFERRALS/ADMISSION  CERTIFICATION 
BETWEEN  DIAGNOSIS  GROUPS, 
CONTROLLING  FOR  GEOGRAPHIC  REGION 


Northeast 

South 

North 
Central 

West 

PSRO  Area 
Subtotals 

High 

Frequency 
Group 

Mean 
+  S.D. 

(N) 

0.114 
+0.178 
(306) 

0.059 
+0.064 
(108) 

0.060 
+0.064 
(126) 

0.071 
+0.149 
(216) 

0.085 
+0.145 
(756) 

Low 

Frequency 
Group 

Mean 
+  S.D. 

(N) 

0.083 
+0.161 
(476) 

0.037 
+0.045 
(168) 

0.039 
+0.045 
(196) 

0.056 
+0.123 
(336) 

0.061 
+0.126 
(1176) 

Geographic 

Region 

Subtotal 

Mean 
+  S.D. 
(N) 

0.095 
+0.169 
(782) 

0.045 
+0.051 
(276) 

0.047 
+0.054 
(322) 

0.062 
+0.134 
(552) 

0.0704 
+0.1339 
( 1932) 

TWO-WAY  ANALYSIS  OF  VARIANCE 

Significance 
F  of  F 


(Main  Effects)  (16.128)  (0.001) 

Region  16.560  0.001 

Diagnosis  Group  14.831  0.001 

Interactive  Effects  0.393  0.999 
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TABLE  88 

DIFFERENCES  IN  ADMISSION  DENIALS/ADMISSION  CERTIFICATION 
BETWEEN  DIAGNOSIS  GROUPS, 
CONTROLLING  FOR  GEOGRAPHIC  REGION 


Northeast 

South 

North 
Central 

West 

PSRO  Area 
Subtotals 

High 

Frequency 
Group 

Mean 
+  S.D. 
(N) 

0.003 
+0.008 
(306) 

0.001 
+0.004 
(108) 

0.000 
+0.001 
(126) 

0.002 
+0.009 
(216) 

0.002 
+0.007 
(756) 

Low 

Frequency 
Group 

Mean 
+  S.D. 
(N) 

0.001 
+0.004 
(476) 

0.000 
+0.001 
(168) 

0.000 
+0.000 
(196) 

0.000 
+0.003 
(336) 

0.001 
+0.003 
(1176) 

Geographic 

Region 

Subtotal 

Mean 
+  S.D. 
(N) 

0.002 
+0.006 
(782) 

0.000 
+0.002 
(276) 

0.000 
+0.001 
(322) 

0.001 
+0.006 
(552) 

0.0011 
+0.0050 
( 1932) 

TWO-WAY  ANALYSIS  OF  VARIANCE 
F 


Significance 
of  F 


(Main  Effects)  (15.265) 

Region  13.604 

Diagnosis  Group  20.250 

Interactive  Effects  1.130 


(0.001) 
0.001 
0.001 
0.336 
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TABLE  89 

DIFFERENCES  IN  CONTINUED  STAY  REVIEW  REFERRALS/CASES  REVIEWED 

BETWEEN  DIAGNOSIS  GROUPS, 
CONTROLLING  FOR  GEOGRAPHIC  REGION 


Northeast 

South 

North 
Central 

West 

PSRO  Area 
Subtotals 

High 

Frequency 
Group 

Mean 
i    +  S.D. 
(N) 

0.434 
+0.243 
(272) 

0.381 
+0.135 
'  (96) 

0.442 
+0.288 
(112) 

0.262 
+0.252 
(192) 

0.379 
+0.253 
(672) 

Low 

Frequency 
Group 

Mean 
+  S.D. 

(N) 

0.383 
+0.260 
(510) 

0.325 
+0.147 
(180) 

0.405 
+0.299 
(210) 

0.212 
+0.256 
(360) 

0.329 
+0.264 
(1260) 

Geographic 

Region 

Subtotal 

Mean 
+  S.D. 

(N) 

0.401 
+0.255 
(782) 

0.344 
+0.145 

(276) 

0.418 
+0.295 
(322) 

0.229 
+0.255 
(552) 

0.3464 
+0.2615 
(1932) 

TWO-WAY  ANALYSIS  OF  VARIANCE 

Significance 

F  of  F 

(Main  Effects)  (50.652)  (0.001) 

Region  61.850  0.001 

Diagnosis  Group  17.057  0.001 

Interactive  Effects  0.082  0.999 
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TABLE  90 

DIFFERENCES  IN  CONTINUED  STAY  REVIEW  DENIALS/CASES  REVIEWED 
BETWEEN  DIAGNOSIS  GROUPS, 
CONTROLLING  FOR  GEOGRAPHIC  REGION 


Northeast 

South 

North 
Central 

West 

PSRO  Area 
Subtotals 

High 

Frequency 
Group 

Mean 
+  S.D. 
(N) 

0.045 
+0.045 
(272) 

0.017 
+0.019 
"  (96) 

0.022 
+0.043 
(112) 

0.029 
+0.042 
(192) 

0.032 
+0.043 
(672) 

Low 

Frequency 
Group 

Mean 
+  S.D. 
(N) 

0.028 
+0.041 
(510) 

0.009 
+0.012 
(180) 

0.015 
+0.037 
(210) 

0.015 
+0.040 
(360) 

0.019 
+0.038 
( 1260) 

Geographic 

Region 

Subtotal 

Mean 
+  S.D. 
(N) 

0.034 
+0.043 
(782) 

0.012 
+0.015 
(276) 

0.018 
+0.039 
(322) 

0.019 
+0.041 
(552) 

0.0238 
+0.0400 
( 1932) 

TWO-WAY  ANALYSIS  OF  VARIANCE 


Significance 
of  F 


(Main  Effects)  (36.211)  (0.001) 

Region  31.048  0.001 

Diagnosis  Group  51.701  0.001 

Interactive  Effects  1.789  0.145 
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6.4    Project  Officer  Ratings  of  PSRO  Performance 


6.4.1  Objectives 

The  primary  objectives  of  this  section  are  to  provide  a  current 
assessment  of  the  extent  to  which  PSROs  are  implementing  the  various 
components  of  the  program  and  to  determine  how  well  these  are  being 
accomplished  by  the  PSROs  as  a  whole.  As  with  Sections  6.2  and  6.3/ 
the  management  and  activity  measures  are  presented  in  order  to 
provide  a  context  within  which  to  consider  the  preceding  quantita- 
tive analyses  of  PSRO  costs  and  impacts  on  hospital  utilization. 

While  one  would  like  to  test  the  underlying  assumption  that 
there  is  a  relationship  between  PSRO  management  and  review  processes 
and  the  impact  on  hospital  utilization  and  quality  of  care,  that  is 
not  possible  at  this  time.  This  relationship  will  be  tested  when 
the  1978  hospital  utilization  data  become  available. 

6.4.2  Methodology 

The  instrument  used  to  provide  project  officer  ratings  consists 
of  56  questions,  most  of  which  have  five  possible  responses. 
Questions  are  grouped  in  nine  major  categories.  Weights  were 
assigned  on  the  basis  of  relative  importance  as  perceived  by 
HSQB  staff.  The  items  in  the  instrument  address  all  of  the  major 
topics  addressed  in  the  on-site  management  assessments  described 
previously,  plus  a  few  items  relevant  to  current  policy. 

The  questionnaire  was  completed  by  the  regional  project 
officer  for  each  of  107  PSROs  which  had  conducted  hospital  review 
for  a  portion  or  all  of  calendar  year  1977.  The  regional  office 
project  officers  were  selected  because  of  their  day-to-day  contact 
with  the  PSROs.  The  ratings  were  based  on  the  individual's  knowl- 
edge of  the  PSRO  during  the  past  year  and  a  half.  In  those  cases 
where  there  were  new  project  officers,  the  individual  most  familiar 
with  the  PSRO  did  the  rating.  There  were  41  raters  in  all;  each 
rated  from  one  to  six  PSROs.     All  107  ratings  were  received. 

Three  of  the  56  questions  were  eliminated  due  to  large  numbers 
of  nonresponses .  For  those  items  which  had  only  a  few  (less 
than  ten)  instances  of  missing  data,  the  average  score  was  assigned. 

Several  approaches  were  undertaken  to  assess  the  data  reli- 
ability. Ranges  and  average  scores  for  each  rater  and  each  HEW 
region  were  assessed  for  marked  variation.  Also,  the  ratings  of  the 
project  officers  were  compared  with  the  26  ratings  done  by  the 
site  assessment  coordinators.  While  the  ratings  of  the  project 
officers  tended  to  be  higher  for  the  26  PSROs,  the  two  sets  of 
rankings  were  comparable. 
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Although  the  longer  range  objective  is  to  test  some  hypo- 
thesized relationships  among  the  management  processes  and  the 
utilization  impact  measures,  that  is  not  feasible  at  this  time. 
Initially  an  effort  was  made  to  assess  the  relative  contribution  of 
categories  of  questions  to  the  overall  rating.  It  was  hypothesized 
that  high  ratings  in  the  following  areas  would  be  related  to  high 
overall  performance  of  a  PSRO:  responsiveness  and  innovativeness 
(questions  40-46);  aggressive  monitoring  of  hospitals  and  follow- 
through  with  appropriate  sanctions  (questions  24/  26,  36-39); 
dynamic  structure  capable  of  taking  effective  actions  (questions  2, 
7,  19,  24,  26,  36-46);  use  of  the  data  system  (questions  13-16,  40, 
46);  and  the  level  of  PSRO-physician  and  PSRO-hospital  interaction 
(questions  2,  24-27,  46). 

Frequency  distributions  and  average  scores  were  calculated 
for  each  question.  Subtotals  and  grand  totals  were  calculated  and 
correlations  were  generated.  At  the  time  of  this  writing,  the  data 
have  not  been  analyzed  as  extensively  as  they  deserve  due  to  pres- 
sures of  time.     Additional  analyses  are  planned. 

6.4.3  Data  Limitations 

This  is  the  first  time  that  the  questionnaire  has  been  used 
and  many  methodological  issues  remain  to  be  studied  and  modifica- 
tions made.  The  assignment  of  weights,  plus  the  unequal  number  of 
items  in  the  catgories,  introduce  a  number  of  potential  problems 
which  have  not  been  adequately  studied  at  the  time  of  writing,  given 
the  time  frame  under  which  the  assessment  has  been  conducted. 
Preliminary  analyses  show  no  significant  difference  in  overall 
ratings  between  weighted  and  unweighted  scores.  A  series  of 
methodological  studies  is  planned  which  will  contribute  to  a  refined 
instrument  for  next  year's  evaluation  report.  Since  this  is  not  a 
tested  measurement  instrument,  one  should  not  treat  the  numerical 
scores  as  absolute  values,  but  rather  as  relative  indicators  of 
levels  of  performance. 

6.4.4  Findings 

6.4.4.1  General 

The  average  score  received  by  the  PSROs  was  98,  with  an 
observed  range  of  47  to  173,  within  a  maximum  possible  range  of 
minus  25  to  plus  219.  Table  91  shows  that  the  total  scores  approach 
a  normal  distribution  but  are  skewed  somewhat  to  the  lower  range  of 
scores.  Table  92  displays  the  average  scores,  as  well  as  the  actual 
and  maximum  possible  ranges,  for  the  total  scores  and  for  each 
separate  category. 

As  pointed  out  in  the  discussions  of  the  site  visit  assess- 
ments (Section  6.2),  there  are  few  outstanding  PSRO  ratings  for 
either  the  total  scores  or  the  subscores.     The  categories  "other 
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TABLE  91 

1978  PROJECT  OFFICER  RATINGS  FOR  PSROs 
CONDUCTING  HOSPITAL  REVIEW  IN  1977 


Number  of 

Total  Score 

PSROS 

Percent 

180  and  above 

0 

0 

160-179 

2 

1.9 

140-159 

2 

1.9 

120-139 

16 

15.0 

100-119 

35 

32.7 

80-99 

21 

19.5 

60-79 

23 

21.5 

40-59 

8 

7.5 

39  or  lower 

0 

0 

TABLE  92 

AVERAGES  AND  RANGES  FOR  TOTAL  SCORES  AND  SUBSCORES 


Maximum 

Average 

Actual 

Possible 

Measure 

Score 

Range 

Range 

Organizational  Strength 

11 

(-1  to  20) 

(-5  to  19) 

General  Management 

28 

(8  to  51) 

(-5  to  53) 

External  Relationships 

0.8 

(-9  to  2) 

(-13  to  2) 

Concurrent  Review 

15 

(5  to  29) 

(-2  to  32) 

Medical  Care  Evaluation  Studies 

9 

(0  to  18) 

(0  to  20) 

Other  Review  Activities 

3 

(0  to  12) 

(0  to  14) 

Denials,  Sanctions,  Fraud  and  Abuse 

7 

(0  to  30) 

(0  to  38) 

Responsiveness  of  Review 

18 

(8  to  32) 

(0  to  40) 

Total 

98 

(47  to  173) 

(-25  to  219) 
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review  activities"  and  "denials,  sanctions,  fraud  and  abuse"  are 
seen  to  be  low  scoring  areas  with  respect  to  the  maximum  possible 
score.  These  findings  are  consistent  with  findings  from  both  the 
site  visit  report  analyses  and  the  site  assessment  coordinator 
ratings.  These  scores  also  are  consistent  with  the  process  of  PSRO 
development. 

Analysis  of  individual  item  responses  shows  findings  much  the 
same  as  those  discussed  in  Section  6.2.  A  few  notable  exceptions 
exist,  which  reflect  maturation  of  the  program.  For  example, 
although  the  scores  for  denials/sanctions/fraud  and  abuse  are  very 
low,  the  percentage  of  PSROs  which  are  using  denials  is  increasing, 
as  are  the  incidences  of  sanctions  initiated  against  hospitals  or 
physicians . 

Unlike  the  analysis  of  the  site  assessment  data,  the  ratings 
reported  in  this  section  reflect  greater  evidence  of  regular  staff 
training.  Only  12%  of  the  107  PSROs  were  reported  as  rarely  pro- 
viding staff  training.  Forty-five  percent  were  reported  as  having 
regular  but  incomplete  training  and  43%  as  having  regular  and 
complete  training. 

As  did  the  site  assessments,  the  project  officer  ratings 
reflect  high  levels  of  PSRO  staff  competence  (67%  rated  as  good  or 
excellent,  20%  average).  Only  about  10  to  12  percent  of  the  PSROs 
were  rated  as  fair  or  poor.  Financial  management  was  also  highly 
rated,  with  again  only  12%  rated  as  poor  or  fair. 

Both  level  of  physician  membership  and  level  of  physician 
actual  participation  appear  to  have  improved  with  time.  Twenty 
percent  of  the  PSROs  were  reported  as  having  "very  high"  actual 
participation  (over  20%)  and  34  percent  were  reported  as  having 
"high"  (10-20%)  actual  participation.  Physician  participation  in 
committees  was  reported  as  average  for  45  percent  of  the  107  PSROs 
and  high  or  very  high  for  40  percent.  About  10-15  percent  of  the 
PSROs  were  rated  as  having  low  or  no  physician  participation  in 
committees  or  policy  making  activities. 

The  site  assessment  reports  discussed  in  Section  6.2  focused  on 
PSRO  problems  with  data  collection,  processing  and  utilization, 
while  generally  noting  the  data  system  capabilities.  The  general 
system  capabilities  currently  range  from  excellent  to  poor  reflect- 
ing the  length  of  experience  of  the  PSROs.  Fifty  percent  of  the 
PSRO  data  systems  were  rated  as  good  or  excellent,  eighteen  percent 
average  and  32  percent  poor  or  fair.  About  20  percent  of  the  107 
PSROs  were  rated  as  low  or  "none"  in  the  areas  of  data  validation 
and  timeliness,  and  fair  or  poor  in  completeness  of  reports.  Thirty- 


279 


three  percent  of  the  PSROs  were  rated  as  having  low  or  no  use  of 
data  outputs  in  the  review  system,  34  percent  average  and  33  percent 
as  having  good  or  excellent  usage  of  data  outputs.  However,  only 
eleven  percent  of  the  107  PSROs  were  rated  as  effectively  using  the 
data  system  to  identify  utilization  problems  or  patterns  of  care. 
Forty-four  percent  were  rated  as  lacking  data  or  reports  sufficient 
for  drawing  conclusions  and  45  percent  lacked  the  resources  or 
capability  to  use  the  data  the  systems  produce. 

The  PSROs  were  reported  as  having  generally  good  relations 
with  Medicare  fiscal  intermediaries/  hospitals,  and  Health  Systems 
Agencies.  Again,  only  about  10-15  percent  of  the  107  PSROs  were 
rated  as  having  poor  or  fair  relations  with  external  organizations. 

Prior  sections  addressed  the  deficiencies  of  PSRO  MCE  activi- 
ties. The  project  officer  responses  confirm  these,  but  indicate 
that  the  situation  is  improving,  as  is  also  indicated  in  Section  5. 
Fifty  percent  of  the  107  PSROs  were  reported  as  performing  95-100 
percent  of  the  required  MCEs  in  1978.  A  quarter  of  the  PSROs  were 
reported  as  having  done  no  areawide  MCEs,  and  half  as  having  con- 
ducted one  or  two.  Only  seven  PSROs  conducted  more  than  five 
areawide  MCEs  in  1978.  Seventy-six  percent  of  the  PSROs'  MCE  topics 
rated  in  appropriateness  as  average  or  above,  24  percent  were  rated 
as  poor  or  fair. 

Little  review  of  ancillary  services  exists,  with  only  two  PSROs 
reported  as  having  fully  implemented  a  system  for  such  review.  Ten 
demonstrations  were  funded  by  HSQB  and  are  developing  review  sys- 
tems.    (See  section  7.3.5.3  for  further  discussion.) 

6.4.4.2  Correlations 

Correlations  were  generated  for  each  subcategory  and  the 
total  as  well  as  among  the  subscores  to  see  how  component  scores 
contributed  to  total  score  and  to  explore  the  interrelationships 
among  subscores. /1  Four  additional  subscores  and  sets  of  correla- 
tions were  generated  based  on  hypothesized  relationships.  Several 
subscores  were  observed  to  contribute  particularly  strongly  to 
predicting  total  score;  however,  all  subscores  contributed  signifi- 
cantly. Table  93  presents  the  correlations  between  subscores  and 
the  total. 


/1     For  technical  reasons,   it  was 'necessary  to  eliminate  fourteen 
PSROs  from  the  correlation  analysis. 
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TABLE  93 

CORRELATIONS  BETWEEN  SUBSCORE  AND  TOTAL  SCORE 


Number  of 

Significance 

Subscore 

Questions 

Correlation 

Level 

Organization 

6 

.587 

.00001 

General  Management 

12 

.835 

.00001 

External  Relationships 

3 

.500 

.00001 

Concurrent  Review 

6 

.581 

.00001 

MCEs 

3 

.478 

.00001 

Other  Review  Activities 

3 

.600 

.00001 

Denials  Sanctions  Fraud  &  Abuse 

4 

.574 

.00001 

Responsiveness  of  Review 

7 

.793 

.00001 

Measures  for  Aggressiveness 

6 

.641 

.00001 

Measures  for  Dynamic  Characteristics 

16 

.900 

.00001 

Use  of  Data 

7 

.819 

.00001 

PSRO/Physician  and  Hospital 

Interaction 

6 

.613 

.00001 

The  strongest  relationships  exist  between  the  total  score  and 
subscores  for  "dynamic  characteristics"  (.900),  "general  manage- 
ment" (.835),  "use  of  data"  (.819),  and  "responsiveness  of  review" 
(.793).  No  extremely  weak  relationshps  are  apparent  from  these 
groupings  of  subscores.  The  correlations  between  subscores  (not 
shown  above)  showed  the  strongest  relationship  (.672)  between 
"general  management"  and  "responsiveness  of  review". 

6.4.4.2.1     Analysis  by  Age 

Differences  were  expected  in  PSRO  scores  and  correlations 
based  on  age.  Age  was  examined  by  separating  PSROs  with  more  than 
18  months  of  review  as  of  January  1,  1978  from  the  newer  PSROs.  The 
average  total  score  for  the  older  PSROs  was  99.2;  the  newer  PSROs 
had  an  average  score  of  97.3. 

In  general,  the  subscores  correlated  more  strongly  with  the 
total  for  the  older  PSROs.  This  was  particularly  true  for  the 
categories  of  concurrent  review;  other  review  activities;  denials, 
sanctions,  fraud  and  abuse;  measures  of  aggressiveness;  use  of  data; 
and  PSRO/physician  and  hospital  interaction.  The  newer  PSROs  have  a 
higher  correlation  for  external  relationships  and  general  manage- 
ment. The  difference  in  correlations  for  general  management  is  very 
small  and  does  not  hold  when  items  relating  to  financial  management 
are  removed  from  the  subscore.  Table  94  contrasts  the  correlations 
by  PSRO  age. 
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TABLE  94 


CORRELATIONS  BETWEEN  SUBSCORES  AND  TOTAL  SCORES  BY  PSRO  AGE 


PSROs  with 

PSROs  with  18 

more  than  18 

months  of  review 

months  of  review 

or  less 

Organization 

.47 

.41 

General  Management 

.84 

.85 

External  Relationships 

.43 

.64 

Concurrent  Review 

.66 

.51 

MCEs 

.50 

.46 

Other  Review  Activities 

.63 

.54 

Denials  Sanctions  Fraud  &  Abuse 

.60 

.50 

Responsiveness  of  Review 

.81 

.75 

Measures  for  Aggressiveness 

.67 

.57 

Measures  for  Dynamic  Characteristics 

.91 

.86 

Use  of  Data 

.85 

.74 

PSRO  Physician  and  Hospital  Interaction  .70 

1 

.53 

The  maturation  process  appears  to  be  occurring  in  PSROs  as  reflected 
in  Table  94,  especially  in  the  review  components,  in  the  respon- 
siveness and  aggressiveness  of  review,  and  in  data  usage.  A  marked 
difference  is  noted  as  well  with  respect  to  physician  and  hospital 
interaction. 
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7.      1978  PROGRAM  STATUS  AND  DIRECTIONS 


7.1  Introduction 

This  section  relates  to  several  of  the  evaluation  issues 
identified  in  the  Program  Evaluation  Plan:  Professional  Standards 
Review  Organizations »/1    These  include: 

1.  Have  the  PSRO  program  elements  (PSROs,  support  centers, 
Statewide  Councils)  been  developed  and  placed  into 
operation  as  planned? 

a.  What  is  the  status  of  the  program  in  terms  of  cover- 
age (geographic  and  beneficiary  population),  basic 
implementation  schedule  and  cost? 

b.  How  far  advanced  is  the  PSRO  program  in  terms  of 
the  progress   that  has  been  made   toward  implementing 

the  PSRO  functions?  ' 

i 

2.  Are  all  Medicare,  Medicaid,  and  Maternal  and  Child  Health 
beneficiaries  covered  by  PSRO  activities? 

a.  Have  PSRO  activities  been  implemented  in  all  hospitals 
which  participate  in  the  Federal  benefit  programs 
and  which  are  located  in  PSRO  areas  with  conditional 
or  operational  PSROs? 

b.  Are  all  Federal  beneficiary  hospital  episodes  receiv- 
ing PSRO  review? 

3.  How  effectively  and  efficiently  are  the  various  PSROs 
organized  in  terms  of  their: 

a.  relationship  with  other  organizations 

b.  staffing 

c.  delegation  of  functions 

d.  review  activity  reimbursement  mechanism 

e.  review  activity 

4.  Has  physician  and  hospital  participation  in  the  Federal 
programs  changed  significantly? 

5.  What  impact  have  organizational  and  environmental  fac- 
tors had  on   changes   in  physician  and  hospital  behavior? 


/1  USDHEW,  Office  of  the  Assistant  Secretary  for  Health,  Office 
of  Professional  Standards  Review,  Program  Evaluation  Plan:  Pro- 
fessional Standards  Review  Organizations  (September  1975),  pp. 
23,  76,  80,  84,  90. 
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These  questions  are  addressed  by  a  presentation  of  discus- 
sions and  data  concerning  the  continued  implementation  and  experi- 
ence of  the  PSRO  program  into  1978.  In  addition,  this  section 
discusses  program  efforts  to  improve  performance. 

The  information  contained  in  this  section  was  obtained  through 
discussions  with  HSQB  personnel  and  review  of  HSQB  documents  and 
memoranda.  It  covers  all  PSROs  in  the  program  and  provides  data 
to  as  recent  a  point  in  1978  as  possible.  The  information  should 
(1)  provide  a  context  within  which  to  view  the  performance  evalua- 
tion in  this  report,  (2)  facilitate  understanding  of  PSRO  activi- 
ties, and  (3)  indicate  program  directions  in  fiscal  year  1979. 
Caution  is  advised  in  one  respect  —  the  data  in  this  chapter 
are  1978  data  while  the  utilization  data  in  the  primary  evaluation 
analyses  are  1977  data.  In  this  respect,  the  two  data  sets  may 
not  be  correlated. 

This  section  is  divided  into  two  general  areas  —  (1)  a  dis- 
cussion of  the  program's  development  over  the  past  year  and  its 
current  implementation  status  (Section  7.2)  and  (2)  a  discussion 
of  program  initiatives  for  improvement  (Section  7.3).  Appendix  I 
of  this  report  contains  a  table  presenting  selected  descriptive 
variables  of  the  individual  PSROs  in  order  to  provide  a  basis 
to  interpret  individual  PSRO  performance. 

7.2    Program  Implementation  Status 

7.2.1     National  Professional  Standards  Review  Council  ( WPSRC) 

The  reorganization  of  DHEW  had  some  effect  on  the  NPSRC. 
Since  the  beginning  of  its  operations,  the  Council  had  been  within 
the  Office  of  the  Assistant  Secretary  for  Health,  Public  Health 
Service  (PHS).  However,  when  the  Health  Care  Financing  Administra- 
tion (HCFA)  was  created  in  March  1977,  it  was  given  responsibility 
for  the  PSRO  program.  This  caused  responsibilities  to  be  somewhat 
divided.  Staffing  and  support  for  the  Council  were  placed  in 
PHS  while  operation  of  the  program  was  placed  in  HCFA.  In  order  to 
end  that  division  of  responsibility  and  to  provide  the  program  with 
clear  and  central  administrative  direction,  the  Assistant  Secretary 
for  Health  formally  transferred  the  function  of  serving  as  Executive 
Secretary  to  the  Council  from  PHS  to  HCFA  on  June  2,  1978.  As 
part  of  the  reorganization,  the  Public  Health  Service  was  requested 
to  maintain  representation  at  all  future  Council  meetings.  Since 
the  transfer,  the  Director  of  HCFA's  Health  Standards  and  Quality 
Bureau  has  begun  the  role  of  Executive  Secretariat  to  the  Council. 
HCFA  has  also  provided  the  NPSRC  with  a  new  Staff  Director. 

During  the  past  year,  the  Council  has  dealt  with  numerous 
quality  and  utilization  issues.  Among  its  recent  major  actions, 
the  NPSRC  has  approved  surgical  criteria  sets  for  concurrent  review, 
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provided  for  a  contract  to  update  the  American  Medical  Association's 
criteria  sets  for  diagnoses  and  surgical  procedures,  established  a 
mechanism  to  provide  technical  assistance  to  PSROs  with  deficient 
criteria,  taken  a  formal  stance  against  a  DHEW  nationwide  program 
for  second  surgical  opinions,  approved  guidelines  for  long  term 
care,  and  expanded  hospital  review  to  include  outpatient  and  emer- 
gency departments  and  ancillary  services. 


7.2.2  Statewide  Professional  Standards  Review  Councils 

The  PSRO  law  requires  the  establishment  of  Statewide  Profes- 
sional Standards  Review  Councils  (SPSRCs)  in  states  where  there  are 
three  or  more  PSROs.  The  Council  appointment  process  begins  when  at 
least  three  PSROs  in  the  state  become  conditional.  With  the  most 
recent  area  redesignations,  sixteen  states  are  required  to  have 
SPSRCs.  By  July  1977,  six  SPSRCs  had  been  contracted:  California, 
Connecticut,  Maryland,  Massachusetts,  New  York,  and  Pennsylvania. 
As  of  October  1978,  Statewide  Councils  have  also  been  contracted  in 
Missouri  and  New  Jersey.  SPSRCs  are  pending  appointment  in  Florida, 
Illinois,  Indiana,  Michigan,  North  Carolina,  Ohio,  and  Virginia.  In 
the  one  remaining  state,  Louisiana,  the  appointment  process  has 
not  yet  begun. 

Of  the  established  SPSRCs,  activities  have  included  setting 
up  the  administrative  and  organizational  structure,  establishing 
Statewide  Council  Advisory  Groups,  and  hearing  appeals.  Five  of  the 
first  six  Statewide  Councils  have  established  Advisory  Groups; 
Connecticut  has  not  yet  done  so.  Limited  information  from  the 
annual  reports  of  those  five  SPSRCs  indicates  that  a  total  of  43 
appeals  had  been  heard  as  of  April  1978.  Individual  Councils  heard 
from  3  to  22  cases.  Thirty-two  decisions  were  upheld,  five  were 
reversed,  and  six  were  still  pending. 


7.2.3  Statewide  PSRO  Support  Centers 

Statewide  PSRO  Support  Centers  were  developed  in  thirteen 
states  to  provide  techical  and  administrative  assistance  to  groups 
interested  in  obtaining  PSRO  funding.  Federal  funding  for  the 
nine  Support  Centers  that  remained  as  of  July  1977  has  been  phased 
out.  These  organizations  still  remain  in  some  states,  but  are  not 
Federally  funded.  In  some  instances,  PSROs  purchase  services  from 
these  organizations. 

7.2.4  Area  Redesignations 

Originally,  the  Secretary  of  HEW  designated  20  3  PSRO  areas 
throughout  the  country.     In  the  year  prior  to  October  1,  1978,  there 
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were  several  area  re designations.  Prevailing  medical  service  pat- 
terns and  other  factors  in  Minnesota  were  considered  in  redesignat- 
ing the  areas  for  that  state.  As  a  result,  the  Secretary  redesig- 
nated the  original  PSRO  Areas  1  and  2  to  be  the  new  Area  1  and  the 
original  Area  3  to  be  the  new  Area  2.  Similar  factors  prompted 
area  designations  for  Ohio  to  be  changed  from  twelve  areas  to 
thirteen.  The  original  nine  Texas  PSROs  have  been  consolidated  into 
one  statewide  area.  These  re designations  bring  the  total  number  of 
PSRO  areas  to  195  throughout  the  nation.  Table  95  displays  the 
number  of  PSROs  by  state.  The  re designations  in  Minnesota  and  Texas 
to  fewer  than  three  PSRO  areas  each  removes  the  requirement  to 
establish  Statewide  Councils  in  these  states. 

Various  refinements  of  other  area  designations  have  been 
made  and  are  being  considered  at  this  time.  However,  these  do  not 
affect  the  total  number  of  areas.  Examples  of  such  refinements 
include  the  redesignation  of  a  county  from  one  PSRO  area  in  a 
state  to  another  PSRO  area  in  the  same  state  or  the  assignment  of  a 
previously  unas signed  zip  code  to  an  existing  PSRO  area  (as  in 
Los  Angeles  County,  California). 

7.2.5    Program  Funding 

The  PSRO  program  is  financed  completely  by  the  Federal  Govern- 
ment. Until  FY  1976,  the  costs  of  nondelegated  PSRO  hospital 
review  activities  were  funded  through  direct  appropriations  and 
the  cost  of  delegated  hospital  review  activity  was  funded  through 
hospital  benefit  payments.  Amendments  to  the  original  law  in 
December  1975  (P.L.  94-182)  provided  for  the  financing  of  both 
delegated  and  nondelegated  review  activities  directly  from  Medi- 
care Trust  Funds.  Management  and  nonhospital  review  costs  (e.g., 
long  term  care  and  ambulatory  care  review)  continue  to  be  financed 
through  direct  appropriation. 


TABLE  96 
PROGRAM  FUNDING/ 1 


Periods 

Total 
Program  Level 

Hospital 
Review  Costs 

Total 
Appropriations/ 2 

FY  1973 

4,475,000 

4,475,000 

FY  1974 

32,850,000 

32,850,000 

FY  1975 

36,208,000 

36,208,000 

FY  1976 

47,645,000 

47,645,000 

Transitional 

quarter 

11,977,000 

11,977,000 

FY  1977 

102,125,000 

41,000,000 

62,000,000 

FY  1978 

147,234,000 

75,000,000 

72,234,000 

FY  1979 

149,866,000 

85,000,000 

64,866,000 

/1  USDHEW,  PSRO  Fact  Book,   1978,  unpublished  draft. 


/2    Total    appropriations    are    actual;    review   costs    are  estimates. 


286 


TABLE  95 
NUMBER  OF  PSRO  AREAS ,   BY  STATE 


FINAL 

FINAL 

STATE 

AREAS 

STATE 

AREAS 

Alabama 

1 

Nebraska 

1 

Alaska 

1 

Nevada 

1 

Arizona 

2 

New  Hampshire 

1 

Arkansas 

1 

New  Jersey 

8 

California 

28 

New  Mexico 

1 

Colorado 

1 

New  York 

17 

Connecticut 

4 

North  Carolina 

8 

Delaware 

1 

North  Dakota 

1 

District  of  Columbia 

1 

Ohio 

13 

Florida 

12 

Oklahoma 

1 

Georgia 

1 

Oregon 

2 

Hawaii,  American  Samoa, 

Pennsylvania 

12 

Northern  Marianas, 

Puerto  Rico 

1 

Guam,  and  the  Trust 

Rhode  Island 

1 

Territories  of  the 

South  Carolina 

1 

Pacific  Islands 

1 

South  Dakota 

1 

Idaho 

1 

Tennessee 

2 

Illinois 

8 

Texas 

1 

Indiana 

7 

Utah 

1 

Iowa 

1 

Vermont 

1 

Kansas 

1 

Virgin  Islands 

1 

Kentucky 

1 

Virginia 

5 

Louisiana 

4 

Washington 

1 

Maine 

1 

West  Virginia 

1 

Maryland 

7 

Wisconsin 

2 

Massachusetts 

5 

Wyoming 

 I 

Michigan 

10 

Minnesota 

2 

Total 

195 

Mississippi 

1 

Missouri 

5 

Montana 

1 
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7.2.6    Review  Implementation 


Table  97  presents  regional-level  data  concerning  the  imple- 
mentation of  PSRO  review.  More  specific  information  is  provided 
later  in  this  section  and  in  Appendix  I. 

7.3    Program  Initiatives 
7.3.1  Background 

As  a  result  of  the  1977  OPEL  evaluation  and  a  study  by  the 
General  Accounting  Office  (GAO) ,/1  the  PSRO  program  has  been  the 
subject  of  concern  by  Congress,  the  Office  of  Management  and 
Budget,  and  the  Department  of  Health,  Education,  and  Welfare. 
Responding  to  Congressional  inquiries,  in  testimony  before  the 
House  Ways  and  Means  Subcommittee  on  Oversight,  Helen  L.  Smits, 
M.D.,  Director  of  the  Health  Standards  and  Quality  Bureau  (HSQB) 
stated,  "The  OPEL  evaluation^  was  particularly  helpful  in  our  re- 
assessment of  the  program."  Accordingly,  HSQB  has  established 
a  set  of  initiatives  to  improve  program  performance.  The  initia- 
tives cover  three  areas:  (1)  program  implementation,  (2)  general 
management  improvements,  and  (3)  review  improvements. 

The  overall  program  represents  a  significant  effort  by  the 
Bureau  to  improve  performance  in  the  PSRO  program.  The  initia- 
tives directly  address  several  areas  of  recognized  deficiency. 
However,  it  is  important  to  realize  that  these  efforts  cannot 
affect  the  1977  utilization  data  that  are  the  primary  basis  for  the 
utilization  impact  and  cost  effectiveness  components  of  this  report 
since  such  efforts  did  not  begin  until  1978.  Any  effect  of  these 
efforts  will  not  be   seen  until    1978   and   1979  data  are  available. 

At  this  time,  it  is  also  difficult  to  assess  the  progress 
of  the  PSRO  program  in  implementing  these  initiatives,  since  the 
target  dates  established  are  primarily  in  1979.  Therefore,  sub- 
jective judgments  must  basically  be  relied  upon  in  order  to  assess 
progress. 


/1  U.S.  Congress,  House  Committee  on  Ways  and  Means,  Gregory 
J.  Ahart,  testimony  before  the  Subcommittee  on  Oversight, 
June  15,  1978. 

/2  U.S.  Congress,  House  Committee  on  Ways  and  Means,  Helen  L. 
Smits,  testimony  before  the  Subcommittee  on  Oversight,  June 
15,  1978. 
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7.3.2    Implementation  Initiatives 


The  initiative  concerning  program  implementation  refers  to 
organizational  funding  and  beginning  of  review  in  hospitals* 
With  regard  to  organizational  funding,  HSQB  has  set  an  objective  of 
having  conditional  PSROs  in  all  195  PSRO  areas  by  March  31,  1979. 
By  the  same  date,  the  Bureau  expects  to  have  4,900  hospitals  under 
review.  As  of  October  1978,  there  were  15  planning  PSROs,  178 
conditional  PSROs,  and  two  unfunded  areas,  Nebraska  and  Illinois 
Area  7.  Tentative  proposals  for  contracts  have  been  received  from 
physician  organizations  in  these  areas,  but  no  contracts  have  been 
negotiated  to  date.  The  15  planning  PSROs  are  expected  to  become 
conditional  sometime  during  FY  1979.  Table  98  displays  the  develop- 
mental stages  of  PSROs  from  1974  to  present. 


TABLE  98 
ORGANIZATIONAL  FUNDING/ 1 


TYPE  OF 
ORGANIZATION 

7/74 

7/75 

7/76 

9/77 

10/78 

4/79 

Planning  PSROs 

91 

58 

33 

62 

15 

0 

Conditional  PSROs 

11 

•  63 

87 

120 

178 

195 

Total 

102 

121 

120 

182 

193 

195 

Unfunded  areas 

101 

82 

83 

21 

2 

0 

Total  areas 

203 

203 

203 

203 

195 

195 

The  funding  of  PSROs  appears  to  be  progressing  satisfactorily 
and  it  appears  that  the  program  will  have  all  areas  funded  by  March 
31,  1979.  Of  a  total  of  4,951  hospitals  in  conditional  PSRO  areas, 
3,442  (70%)  have  implemented  review  as  of  June  30,  1978.  The  goal 
of  4,900  hospitals  performing  review  by  March  31,  1979  represents 
approximately  82  percent  of  all  acute  and  specialty  non-Federal 
hospitals  in  all  PSRO  areas.  On  this  basis,  implementation  of 
review  in  hospitals  will  have  to  be  accelerated  to  meet  the  1979 
goals . 


/1  USDHEW,  PSRO  Fact  Book,  1978,  plus  update  by  HCFA  personnel. 
The  figures  shown  in  the  column  headed  "4/79"  are  HSQB  objec- 
tives . 
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The  original  PSRO  law,  P.L.  92-603,  required  funded  organi- 
zations to  perform  review  on  a  conditional  basis  for  a  trial  per- 
iod not  to  exceed  24  months  prior  to  achieving  full  designation. 
However,  subsequent  amendments  in  P.L.  95-142  extended  the  trial 
period  to  a  maximum  of  48  months  and  allowed  72  months  where  the 
Secretary  finds  that  a  PSRO  has  been  unable  to  perform  satisfac- 
torily for  reasons  beyond  the  organization's  control.  By  the 
end  of  fiscal  year  1979,  it  is  expected  that  5  0  current  conditional 
PSROs  will  be  converted  to  a  fully  designated  status. 

As  part  of  the  program  initiatives  for  1978-1979,  HSQB  intends 
to  end  funding  of  PSROs  that  are  judged  to  be  performing  inade- 
quately in  terms  of  the  operation  of  the  organization  or  effective- 
ness of  review.  Several  organizations  are  expected  to  be  defunded 
by  March  31,  1979.  These  PSROs  would  be  replaced  with  physician 
organizations  or  alternate  PSROs  as  allowed  by  the  law. 

As  a  result  of  a  legislative  amendment,  HSQB  has  converted 
funding  of  individual  PSROs  from  a  contract  basis  to  a  grant  basis. 
As  of  October  1978,  27  PSROs  had  been  converted  to  grants.  The 
remaining  conditional  organizations  should  convert  to  grants  during 
fiscal  year  1979.  The  grant  approach  is  being  used  to  allow  PSROs 
more  flexibility  to  change  programmatic  activities  and  financial 
resources  to  meet  changing  needs  after  the  budget  negotiations 
are  complete. 

7.3.3    General  Management  Initiatives 

General  management  initiatives  in  the  improvement  plan  con- 
sist of  (1)  establishment  of  performance  objectives,  (2)  self- 
assessment  of  impact,  (3)  improved  financial  management,  and  (4) 
improved  relationships  with  external  agencies,  such  as  Health 
Systems  Agencies  and  Medicaid  state  agencies. 

7.3.3.1     Objective  Setting 

Objective  setting  within  HSQB  refers  to  (a)  establishing 
performance  objectives  responsive  to  locally-identified  problem 
areas  and  (b)  working  toward  their  realization.  The  objective 
setting  process  is  intended  as  the  cornerstone  of  each  PSRO's  effort 
to  improve  performance.  Both  long  and  short  range  objectives  are 
included.  The  process  is  based  on  (1)  identification  of  problem 
areas,  (2)  setting  measurable  area-specific  objectives  to  resolve 
these  problem  areas,  (3)  definition  of  methodologies  to  attain  the 
objectives,  and  (4)  initiation  of  corrective  actions  to  solve  the 
problems.  The  process  requires  frequent  evaluation  of  progress, 
assessment  of  barriers  and/or  factors  contributing  to  change,  and 
when  necessary,  redefinition  of  objectives. 
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As  part  of  the  refunding  process,  all  PSROs  are  required 
to  set  measurable  area-specific  objectives  within  the  context 
of  national  concerns.  Areas  for  objectives  include  admission 
rates,  average  length  of  stay,  days  of  care  rates,  and  diagnosis- 
and  procedure-specific  utilization  variables,  aa  well  as  quality- 
related  goals*  Discussion  of  the  objectives  for  the  next  con- 
tract year,  as  well  as  progress  made  toward  meeting  the  current 
year's  objectives,  are  held  between  the  PSRO  and  HSQB.  In  work- 
ing with  PSROs,  HSQB  has  sought  to  emphasize  that  objectives, 
while  being  definitive  goals  against  which  to  measure  perform- 
ance, are  not  intended  to  be  statements  of  required  performance 
to  assure  continued  funding. 

Objectives  can  be  general  or  specific  and  can  be  related 
to  both  management  and  process  as  well  as  to  the  overall  PSRO 
goal  of  assuring  quality  and  appropriate  utilization  of  health 
care  services.  As  PSROs  become  more  efficient  in  data  collection 
and  are  able  to  generate  profiles  of  physicians,  institutions, 
and  diagnoses/procedures  which  identify  problem  areas,  more  specific 
objectives  can  be  set.  Even  PSROs  without  operational  data  systems 
must  set  objectives  using  whatever  information  is  available.  It 
is  important  that  these  be  measurable  so  that  progress  or  lack  of 
progress  can  be  determined  on  a  continuing  basis  (i.e.,  quarterly 
or,  at  a  minimum,  semi-annually). 

PSROs  are  being  provided  technical  assistance  from  HSQB  re- 
gional and  central  office  staff  through  such  mechanisms  as  trans- 
mittals, general  memoranda,  workshops  and  individual  onsite  visits. 
In  addition,  HSQB  is  working  closely  with  twenty  well-established 
PSROs  to  demonstrate  the  process  of  establishing  specific  measurable 
area  objectives  (e.g.,  reducing  specific  physician  and  hospital 
services  which  are  above  area  means,  reducing  high  utilization  rates 
for  certain  diagnoses  by  a  specific  percentage).  Data  bases  such  as 
the  PHDDS  and  Medicare  utilization  data  are  also  being  used  to 
provide  a  statistical  basis  for  objective  setting. 

Objective  setting  should  enable  HSQB  to  monitor  an  individual 
PSRO's  progress  toward  solving  identified  area  problems,  as  well 
as  to  identify  national  concerns.  Also,  evaluation  of  national  and 
individual  PSRO  data  provides  insight  into  how  program  policies  will 
affect  these  national  utilization  and  quality  problem  areas  as  PSROs 
attempt  to  influence  the  necessary  changes. 

The  examples  presented  below  illustrate  objectives  for  several 
utilization  indicators,  as  well  as  an  overall  programmatic  emphasis, 
and  the  steps  which  have  been  identified  as  necessary  to  achieve 
these  objectives.  These  have  been  developed  based  on  samples 
submitted  by  several  PSROs.  It  should  be  kept  in  mind  that  each 
would  be  only  part  of  a  given  PSRO's  total  set  of  objectives  for  the 
coming  year. 


292 


Example  #1 


Identified  Problem:     Days   of   Care    (DOC)    rate   for  area  is 

20  percent  above  the  HEW  regional  rate* 
Regional  rate  is  3,382  days  per  1,000 
enrollees. 

Goal:  To  reduce  the  DOC  rate  in  the  area  by 

10   percent   to   3,65  4  within   12  months. 

Objective:  To  identify  those  providers/practitioners/ 

diagnoses/procedures  which  contribute  to 
the  high  DOC  rate. 

Methodology: 

•  Develop  profiles  on  all  hospitals  for  total  DOC  per  1,000 
enrollees  (TDOC) ,  average  length  of  stay  (ALOS)  and  dis- 
charges per  1,000  enrollees  (DISC)  for  the  diagnoses/pro- 
cedures which  account  for  75  percent  of  admissions. 

•  Set  standards,  adjusted  for  case  mix,  based  on  inter- 
hospital  comparability,   for  each  diagnosis/procedure. 

•  Identify  those  hospitals  which  appear  to  be  above  the 
area  mean  for  each  diagnosis/procedure   in  terms  of  TDOC. 

•  Identify  which  factor  (ALOS  or  DISC)  for  each  hospital 
is  out  of  line  and  thus  contributing  to  high  TDOC. 

•  Establish  the  following  plan  of  corrective  action  with  each 
hospital  contributing  to  high  TDOC: 

Study  of  individual  cases  which  are  twenty  percent  or 
more  above  the  mean  of  specific  diagnoses/procedures 
in  order  to  determine  possible  causes,  explanations, 
etc.,  provide  opportunity  for  discussion  with  specific 
practitioners  and,  when  appropriate,  provide  for  con- 
tinuing education  sessions. 

Assessment/Evaluation : 

•  Analyze  data  for  specific  providers,  practitioners  and 
diagnoses/procedures  on  a  continuing  basis  so  that  quar- 
terly assessments   can  be  made   of  progress   toward  goals. 

•  Monitor  other  providers,  practitioners,  and  diagnoses/ 
procedures,  on  a  sample  basis,  to  ensure  that  no  major 
changes  are  taking  place  which  will  adversely  affect  goal 
attainment . 
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Example  #2 

Identified  Problem:  Profiling  ALOS  of  ten  most  common  diag- 
noses and  procedures  reveals  several 
which  are  above  both  the  regional  and 
national  ALOS. 

Goal:  To  bring  the  ALOS  for  the  following 

diagnosis  and  procedure  to  within  five 
percent  of  the  national  ALOS: 

PSRO's  Nat'l 

Diagnosis/Procedure  ALOS        ALOS  Goal 

Acute  myocardial  infarction  (AMI)  19.3  15.4  16.2 
Cholecystectomy  16.0        13.4  14.1 

To  be  accomplished  within  twelve  months. 

Objective:  To  identify  those  providers /practitioners/ 

diagnoses/procedures  whose  patients 
contribute  to  the  excessive  ALOS  for  the 
above  diagnosis  and  procedure. 

Methodology : 

With  those  hospitals  whose  ALOS  is  above  the  national  average, 
develop  plans   to  study  possible   causes.      By  third  month, 

•  Study  whether  there  are  any  quality  factors  affecting 
LOS  for  AMI  patients,  e.g.,  length  of  time  in  coronary  care 
unit  and  availability  of  appropriate  equipment. 

•  Study  whether  pre-operative  stays  for  cholecystectomy 
are  longer  than  necessary  and  thus  contribute  to  long 
ALOS. 

•  Develop  educational  seminars  to  communicate  problems 
and  possible  solutions. 

•  Plan  to  monitor  on  quarterly  basis  to  insure  changes  are 
being  effected. 

With  other  hospitals,  monitor  quarterly  to  insure  continued 
appropriate  ALOS  for  AMI  and  cholecystectomy  patients. 

Assessment/Evaluation : 

•  Continuously  analyze  data  on  AMIs  and  cholecystectomies 
so  as  to  update  ALOS  on  bi-monthly  basis. 

•  On  quarterly  basis,  check  other  diagnoses/procedures  to 
insure  continued  apropriate  ALOS. 
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Example  #3 

Identified  Problem:     Inefficient  or  incomplete  quality  assess- 
ment  and  quality   assurance  methods. 

Goal:  To  ensure  an  organized  approach  to  quality 

assessment  and  quality  assurance  in  area 
hospitals  on  a  continuing  basis* 

Objectives:  To   conduct  MCE   studies,    as   required,  in 

each  hospital  and  to  ensure  that  correc- 
tive action  has  taken  place  and  continues. 

To  facilitate  sharing  of  MCE  study 
designs,   corrective  action  plans,  etc. 

Methodology: 

m  Work  with  staff  from  ail  hospitals  to  ensure  understanding 
of  MCE  process  and  develop  guidelines  as  appropriate  - 
by  second  month* 

•  Review  study  design  for  all  MCEs  and  draft  reports  of 
findings  and  corrective  action  to  be  taken  after  initial 
study  completed;   comment  and  provide  assistance  as  needed. 

•  Monitor  corrective  action  through  progress  reports  from 
hospitals. 

•  Develop  mechanism  so  that  hospitals  with  common  problems 
can  share  study  designs  and  corrective  action  plans  - 
by  third  month. 

•  Encourage  two  or  more  hospitals  with  common  problems 
to  conduct  multi-hospital  MCEs. 

Assessment/Evaluation : 

•  Set  up  tickler  file  to  ensure  that  re-audits  are  carried 
out  and  evaluate  findings. 

•  Set  up  system  so  that  study  area  is  re-evaluated  six 
months  and  one  year  after  re-audit  completed. 

7.3.3.2    Self -Assessment  of  Impact 

An  important  part  of  individual  PSROs'  efforts  to  improve 
their  performance  is  the  self -assessment  of  impact.  PSROs  have  been 
asked,  as  part  of  their  contract  renewal  application,  to  state  the 
changes  which  have   occurred  within  the  area   during  their  current 
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year  of  operation,  the  major  problems  which  have  been  encountered, 
and,  where  appropriate,  what  is  being  done  to  correct  the  problems. 
Many  PSROs  responded  to  this  request;  others  reported  impact  studies 
separately  from  their  contract  applications.  Three  examples  of 
reported  self -assessments  are  shown  in  Table  99. 

The  scope  and  rigor  of  the  self-assessments  has  been  somewhat 
limited  to  date.  However,  the  analytical  capabilities  of  the 
PSROs  are  increasing  with  improved  technical  assistance  and  data 
processing  functions.  Many  PSROs  are  now  able  to  review  data  for 
both  a  pre-  and  a  post-period  of  PSRO  review  activity.  However, 
there  are  several  factors  which  often  prevent  a  PSRO  from  making 
conclusive  determinations  of  impact.  Frequently,  multiple  sources  of 
data  have  to  be  used  and  exact  comparability  of  those  sources  is  not 
possible  (e.g.,  Medicare  data  vs.  PHDDS  data).  Also,  often  there 
may  not  be  enough  points  in  time  prior  to  and  after  PSRO  implemen- 
tation to  show  any  trends.  As  PSRO  data  become  more  reliable,  the 
documentation  and  presentation  of  changes  by  PSRO  will  be  more 
meaningful.  Self-assessments  are  expected  to  receive  increased 
emphasis  in  1979. 

7.3.3.3    Financial  Management 

Improved  financial  management  at  the  local  level  is  expected 
to  be  a  major  factor  in  controlling  the  costs  of  the  PSRO  program. 
The  primary  goal  of  the  cost  control  initiative  is  to  reduce  the 
unit  cost  of  hospital  review  from  approximately  $13  per  discharge 
to  an  average  of  $8.70  per  discharge  by  September  30,  1979.  Focused 
review  (see  Section  7.3.4.1)  is  the  primary  method  to  be  used  in 
reducing  costs.  However,  the  Bureau  also  plans  to  assist  PSROs  in 
this  regard  by  providing  financial  management  technical  assistance, 
identifying  the  elements  of  financial  management  in  the  most  cost- 
effective  PSROs  which  can  be  of  assistance  to  other  PSROs,  develop- 
ing guidelines  for  administrative  and  support  costs,  and  tightening 
the  auditing  of  delegated  hospitals  to  eliminate  possible  "loading" 
of  non-review  costs  onto  PSROs. 

The  major  cost  cutting  effort  is  directed  at  reducing  the 
costs  of  hospital  review.  Hospital  review  unit  cost  targets  are 
used  to  develop  total  individual  PSRO  budgets  as  opposed  to  the 
traditional  method  of  line-by-line  budget  negotiation.  The  unit 
cost  target  is  partially  determined  according  to  the  PSRO's  Medicare 
days  of  care  (DOC)  rate  in  relation  to  the  national  Medicare  DOC 
rate.  In  addition,  altering  the  funding  mechanism  from  one  of 
contracts  to  a  grants  mechanism  is  expected  to  allow  more  individual 
flexibility  in  the  financial  management  of  PSROs  to  meet  unit  cost 
targets.  Each  PSRO  is  required  to  develop  its  budget,  including 
delegated  review  costs,  such  -that  the  areawide  average  hospital 
review  unit  cost  is  in  line  with  the  PSRO's  unit  cost  target. 

HSQB  is  monitoring  unit  costs  by  PSRO  on  a  quarterly  basis 
for  the  latest  twelve-month  period.     Unit  costs  will  be  calculated 
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for  total  hospital  review  consisting  of  concurrent  review  (CR) , 
MCE  studies,  and  areawide  review,  and  for  CR  and  MCEs  separately  by 
delegation  status.  Also,  PSROs  are  being  asked  to  calculate  their 
own  unit  costs  based  on  a  standardized  formula  as  part  of  their 
application  for  funding. 

7.3.3.4    Improved  Relationsips  with  External  Agencies 

Steps  are  being  taken  to  improve  relationships  with  Health 
Systems  Agencies  (HSAs)  and  third  party  payors,  e.g.,  state  Medicaid 
agencies  and  Medicare  fiscal  intermediaries.  Potential  PSRO  effec- 
tiveness should  be  enhanced  by  improving  such  relationships.  The 
HSAs  can  assist  PSROs  in  maintaining  a  permanent  impact  on  the  total 
cost  of  health  care  by  providing  for  new  long  term  care  beds  where  a 
lack  of  such  beds  has  hindered  PSRO  effectivness  in  the  past  and  by 
closing  unneeded  acute  care  beds.  State  Medicaid  agencies  can 
monitor  PSRO  findings  and  take  into  account  observations  of  sub- 
standard quality  of  care  in  their  certification  processes. 

Monitoring  of  PSRO  review  decisions  by  Medicare  fiscal  inter- 
mediaries is  required  on  a  twenty  percent  sample  of  all  PSRO  re- 
views. Available  information  indicates  that  the  fiscal  intermedi- 
aries, using  various  methods,  reflect  varying  degrees  of  disagree- 
ment with  PSRO  decisions.  The  disagreement  ranges  from  less  than 
five  percent  of  the  sample  cases  to  almost  twenty-five  percent, 
with  most  PSROs  at  the  lower  end  of  this  range.  Where  disagreements 
exist,   the  differences  are  resolved  between  the  two  organizations. 

As  part  of  the  cost  reduction  initiatives  being  implemented 
by  HSQB,  the  costs  of  monitoring  by  fiscal  intermediaries  are 
being  investigated  for  alternative,  less  costly  methods.  Changes 
being  considered  include  dropping  the  twenty  percent  sample  in  favor 
of  monitoring  perceived  problem  areas  and  combining  monitoring 
activities  with  claim  processing  to  eliminate  duplicate  handling. 
Implementation  of  any  revisions  is  not  expected  before  early  1979. 

Public  Law  95-142  amended  the  original  PSRO  law  regarding 
monitoring  of  PSRO  review  of  Medicaid  patients  by  state  agencies. 
The  law  now  permits  state  agencies  to  monitor  such  review  according 
to  a  formal  plan  reviewed  and  commented  upon  by  the  state's  PSRO(s) 
and  approved  by  the  Secretary  of  HEW.  The  guidelines  for  such  plans 
are  quite  flexible  and  no  quantitative  requirements  are  specified. 
As  of  September  1978,  twelve  plans  had  been  submitted,  eight  of 
which  were  approved.  Other  states  are  performing  limited  monitoring 
but  not  under  an  approved  plan.  The  monitoring  of  PSRO  Medicaid 
review  is  in  too  early  a  state  of  development  for  assessment  at  this 
time. 
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7.3.4    Review  Improvements 


Review  improvement  initiatives  concentrate  on  four  areas: 
focused  review,  profile  analysis,  MCE  studies,  and  surgery  evalua- 
tions including  study  of  possible  unnecessary  surgical  utiliza- 
tion and  development  of  model  criteria  sets  for  surgical  review. 
These  initiatives  are  intended  both  to  enhance  impact  on  utiliza- 
tion and  quality  and  to  control  costs. 

7.3.4.1     Focused  Review 

Detailed  concurrent  review  of  all  admissions  is  expensive 
and  of  questionable  potential  relative  to  its  cost.  Review  experi- 
ence has  shown  that  in  many  areas  inappropriate  utilization  is  not  a 
problem.  There  are  also  known  areas  of  inappropriate  utilization 
where  review  resources  could  be  more  effectively  applied.  To  allow 
for  more  efficient  use  of  review  resources,  HSQB  is  taking  steps  to 
expedite  PSRO  implementation  of  a  high  level  of  focused  review  by 
March  31,  1979. 

In  areas  where  inappropriate  utilization  is  not  a  problem, 
focused  review  permits  the  PSRO  to  phase  out  100  percent  review  and 
replace  it  with  monitoring  on  a  sample  or  other  basis.  In  a  known 
problem  area,  a  more  intense  review  system  may  be  employed.  Areas 
for  focused  review  may  be  chosen  on  the  basis  of  diagnosis,  proce- 
dure, provider  institution,  physician,  etc. 

HSQB  funded  twelve  one-year  demonstration  projects  in  the  last 
quarter  of  1977  to  develop  methods  of  focused  review.  Appendix 
I  indicates  which  PSROs  are  involved  in  the  demonstrations.  Also, 
technical  assistance  will  be  provided  to  PSROs  through  workshops, 
technical  assistance  documents  and  other  means  which  become  avail- 
able. 


7.3.4.2    Profile  Analysis 

Difficulties  in  developing  automated  data  systems  in  many 
PSROs  have  prevented  tr la  generation  of  practitioner  and  institu- 
tional profiles  for  profile  analyses.  With  more  PSROs  attaining 
100  percent  implementation,  more  attention  is  being  devoted  to 
data  system  development.  HSQB  expects  approximately  one  hundred 
PSROs  to  have  the  capability  to  produce  and  analyze  profiles  by 
the  end  of  1978  to  assist  in  identifying  patterns  of  unnecessary 
utilization  and  poor  care.  To  accelerate  progress  in  this  area, 
technical  assistance  is  being  provided  by  HSQB  and  contractors  in 
developing  profiling  methods.  All  PSROs  with  data  systems  are 
required  to  develop  minimal  institutional  and  practitioner  profiles. 
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While  all  PSROs  were  required  to  conduct  profile  analysis , 
there  had  been  no  established  reporting  system  to  assess  the  scope 
of  profiling  implementation.  However,  revisions  to  the  Federal 
Reports  Manual  recently  approved  by  OMB  incorporate  reporting 
requirements.  By  July  1978,  forty-two  PSROs  had  performed  some 
profiling  and  reported  PHDDS  data  on  1,104,477  discharges  from 
January  to  June  1978. 

HSQB  has  funded  seven  profiling  demonstration  projects,  some  of 
which  are  joint  demonstrations  with  focusing  projects.  These  PSROs 
are  receiving  special  emphasis  and  attention  in  order  to  develop 
various  methods  of  profile  analysis. 


7.3.4.3  MCEs 

The  1977  PSRO  evaluation  reported  that  performance  of  MCE 
studies  generally  lagged  behind  the  implementation  of  concurrent 
review  by  one  year.  At  that  time,  available  data  indicated  that 
about  half  of  the  conditional  PSROs  had  performed  at  least  one 
study.  Since  that  time,  the  number  of  MCE  studies  reported  has 
increased,    as   discussed   in   Section   5   of  this   evaluation  report. 

A  substantial  number  of  PSROs  have  identified  quality-related 
deficiencies  and  have  focused  efforts  toward  corrective  action 
to  reduce  the  problems.  Overall  deficiencies  appear  to  fall  into 
two  major  areas:  (1)  high  incidence  of  death  rates  and  complication 
rates  and  (2)  inappropriate  use  of  services/procedures.  Table 
100  summarizes  examples  of  the  process  PSROs  undertake  in  addressing 
quality  issues;  the  areas  shown  in  the  table  include  high  mortality 
rate,  high  complication  rate,  and  inappropriate  use  of  services  and 
procedures  respectively. 


7.3.4.4    Surgical  Evaluations 

Owing  to  the  high  cost  of  surgical  care,  unnecessary  surgical 
utilization  has  also  been  the  subject  of  Congressional  concern 
and  HSQB  initiatives.  Model  criteria  for  review  of  eleven  common 
elective  surgical  procedures  have  been  developed  and  approved  by  the 
National  Professional  Standards  Review  Council.  Five  PSROs  with 
operational  data  base  systems  and  surgical  rates  above  the  national 
average  were  chosen  to  participate  in  a  Surgical  Evaluation  Project. 
The  review,  which  is  either  concurrent  or  retrospective  depending 
on  the  PSRO,  was  instituted  at  three  sites  in  the  fall  of  1978  and 
will  begin  at  the  other  two  in  1979. 


300 


INAPPROPRIATE  USE  OF  PROCEDURES 

To  determine  how  many  unnecessary 
hiatal  hernia  repairs  are  performed 
in  area  hospitals. 

1 •  An  areawide  audit  was  conducted 
in  thirteen  hospitals. 

2.  Thirty-four  patients'  records 
were  reviewed  retrospectively 
from  4/76  to  4/77. 

3.  Quality  Assurance  Committee  de-j 
veloped  standards  for  all  patients 
requiring  this  repair  (e.g.,  ex- 
hibit acute  symptoms  such  as  pain, 
bleeding  or  dysphagia;  diagnosis 
to  be  confirmed  by  positive  find- 
ings, e.g.,  GI  or  esophagoscopy). 

1.  One  hospital  lacked  Justifiable 
indications  for  performing  this 
surgery.     Apparent  causes  were: 

a.  lack  of  documentation • in  records 
to  support  diagnosis  or  treat- 
ment. 

b.  indications  of  poor  surgical 
judgment. 

2.  Two  physicians  who  were  partners 
performed  80*  of  these  repairs. 

1.  PSRO  Board  of  Directors  notified 
hospital  of  its  findings  and 
requested  that  deficiencies  be 
corrected. 

2.  Discussions  were  held  with  each  in- 
dividual physician  about  his  know- 
ledge of  hiatal  hernia  repair. 

3.  Deficiencies  were  still  not  cor-, 
rected  upon  a  restudy  six  months 

later. 

4.  Sanction  hearings  have  begun  on 
these  physicians. 

5.  Potential  impact  96  hernia 
patients  annually. 

INAPPROPRIATE  USE  OF  SERVICES 

To  determine    variances  in  the 
practice  of  typing,  cross-matching 
and  use  of  blood  in  surgeries  not 
generally  requiring  hemotherapy 
in  area  hospitals. 

1.  Eight  surgical  procedures  in 
which  hemotherapy  is  not  re- 
quired were  selected  for  this 
study. 

2.  The  criteria  were  developed  by 
a  subcommittee  of  three  path- 
ologists.   The  number  of  units 
ordered  were  compared  to  the 
number  of  units  of  blood  actu- 
ally given. 

3.  Records  of  721  patients  were 
reviewed  to  see  whether  blood 
had  been  given  in  the  operat- 
ing or  recovery  rooms. 

Among  the  thirteen  hospitals 
studied,  there  was  a  marked  vari- 
ance in  the  practice  of  typing, 
cross  matching,  and  use  of  blood 
(e.g./  some  hospitals  routinely 
ordered  blood  on  all  surgical 
patients  while  others  did  not). 

1.  Hospitals  were  required  to  sub- 
mit policies  of  their  blood 
transfusion  procedures  to  the 
PSRO. 

2.  All  hospitals  were  able  to  re- 
view the  results  of  the  study 
and  be  aware  of  where  they 
stood  in  relation  to  other 
hospitals. 

3.  Educational  programs  were  in- 
itiated by  the  hospitals  to 
eliminate  possible  waste  of 
blood. 

4.  A  restudy  began  in  October 
1978. 

5.  Potential  Impact:  8,652  pat- 
ients annually  for  the  pro- 
cedures studied. 

HIGH  COMPLICATION  RATE 

To  determine  why  some  SNFs 
had  a  higher  rate  of  urin- 
ary tract  infection  (UTI) 
than  other  facilities. 
(Problem  identified  through 
concurrent  review. ) 

1.  One  hundred  Federal  and 
non-Federal  patients' 
records  were  reviewed 
retrospectively  between 
3/77  and  2/78. 

1 .  The  occurrence  of  UTI 
patients  appeared  to  be 
almost  routine. 

2.  UTI  usually  followed 
staff  shortages  and  a 
marked  increase  in  the 
number  of  catheters  in- 
serted. 

1.  Assistance  was  provided 
to  the  facility  by  de- 
signing a  patient  care 
evaluation  study  plan. 

2.  Decreases  in  UTI  occurred 
due  to  improved  catheter 
maintenance  procedures, 
intensive  catheter  care 
in-service  training  ses- 
sions, and  consistent 
medical  monitoring  of 
infections. 

3.  UTI  was  reduced  from  21 
patients  in  March  1977  to 
eight  patients  in  Febru- 
ary 1978.   (Two  of  the  21 
patients  had  UTI  prior  to 
admission  to  SNF.) 

4.  Other  SNFs  used  this 
study  to  Identify  pro- 
blems of  UTI  in  their 
facility. 

5.  Potential  impact:  361 
UTI  patients  annually. 

HIGH  MORTALITY  RATE 

To  determine  whether  a  high 
mortal llty  rate  of  acute  my- 
ocardial infarction  (AMI) 
existed  in  area  hospitals. 

1.  Observed  LOS  of  AMI  pat- 
ients in  66  hospitals. 

2.  790  patients'  records  with 
LOS  ranging  from  6.3  days 
to  25.5  days  were  reviewed 
retrospectively. 

3.  PSRO  focused  its  attention 
on  extremely  short  LOS. 

4.  Quality  Assurance  Comraitte 
developed  standards  for  all 
AMI  patients  (e.g.  signs  of 
unstable  angina  pectoris  at 
admission  and  ERG  performed 
to  confirm  diagnosis). 

1.  It  was  observed  that  short 
LOS  corresponds  with  high 
mortality  rates. 

2.  34  hospitals  revealed  a 
high  mortality  rate.  Appar- 
ent causes  were: 

a.  Lack  of  specialty 
training  for  nurses  and 
physicians 

b.  lack  of  appropriate 
equipment 

c.  frequent  admission  to 
end  treatment  of  AMI 
patients  in  hospitals 
with  no  coronary  care 
units. 

1.  Hospitals  with  high  mortal- 
ity rates  were  required  to 
transfer  AMI  patients  to 
hospitals  with  proper  fac- 
ilities. 

2.  Recommended  each  hospital 
institute  an  educational 
program  on  the  diagnosis 
and  treatment  of  AMI. 

3.  Sixty  hospitals  partici- 
pated in  teaching  standard 
coronary  care  procedures. 

4.  Restudy  conducted  12/77 
indicated  AMI  mortality 
rates  reduced  by  15*  to 
30%. 

5.  Potential  impact:  3,350 
AMI  patients  annually. 

STUDY  AREA 

PURPOSE  OF 
STUDY 

[I 

1 

X 

FINDINGS 

ACTIONS  AND 
IMPROVEMENTS 
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7.3.5    Demonstration  Projects 


HSQB  sponsors  a  program  of  PSRO  demonstration  projects  in 
various  subject  areas.  Two  of  these  areas,  focused  review  and 
surgical  evaluations,  have  already  been  discussed.  The  remaining 
demonstrations  are  for  review  of  long  term  care,  ambulatory  care, 
and  ancillary  services.  Generally,  the  state  of  the  art  of  review 
in  these  areas  is  in  its  early  developmental  stages  and  the  pro- 
jects are  experimental  in  some  respects.  It  is  intended  that 
these  projects  will  provide  information  and  experience  on  several 
alternative  methodologies  from  which  departmental  policies  will  be 
developed  with  respect  to  such  review  components.  The  demonstra- 
tions taking  place  in  the  various  PSROs  are  indicated  in  Appendix 
I. 

7.3.5.1    Long  Term  Care  Review 

Public  Law  95-142  amended  Title  XI  of  the  Social  Security  Act 
with  respect  to  long  term  care  review  requirements.  Whereas  before, 
PSROs  had  responsibility  for  review  of  services  in  skilled  nursing 
facilities  (SNFs)  and  intermediate  care  facilities  (ICFs),  the 
states  now  retain  authority  for  review  in  ICFs  (and  public  facili- 
ties for  the  mentally  retarded)  unless  (1)  the  state  fails  to 
perform  effective  review,  (2)  the  state  requests  the  PSRO  to  perform 
the  review,  or  (3)  the  ICF  is  part  of  a  delegated  hospital.  In  the 
instance  of  an  institution  with  both  an  ICF  and  SNF,  DHEW  may 
require  the  PSRO  to  conduct  binding  review  where  the  state's  review 
would  be  inefficient.  PSROs  retain  authority  for  review  of  care  in 
SNFs,  but  may  no  longer  delegate  review  to  the  SNF  unless  it  is  a 
distinct  part  of  a  delegated  hospital. 

In  fiscal  year  1977,  there  were  fifteen  long  term  care  demon- 
stration projects  and  five  PSROs  that  were  authorized  to  perform 
binding  long  term  care  review.  Of  the  fifteen  demonstration  pro- 
jects, the  Vermont  PSRO  performs  review  of  Medicaid  long  term 
care  services  under  a  Section  1115  grant  from  the  Office  of  Policy, 
Planning,  and  Research,  HCFA,  DHEW.  In  fiscal  year  1978,  twenty-two 
additional  PROs  were  funded  for  binding  long  term  care  review. 
These  PSROs  were  required  to  have  the  support  of  the  State  Medicaid 
agency  as  a  prerequisite  for  approval  of  their  plan.  The  total 
number  of  PSROs  currently  performing  long  term  care  review  is 
forty-two.  The  PSROs  performing  binding  long  term  care  review  are 
required  to  review  all  services  in  all  institutions  under  their 
purview.  These  PSROs  primarily  adhere  to  the  "Guidelines  for  PSRO 
Long  Term  Care  Review"  in  PSRO  Transmittal  Number  62  and  long  term 
care  review  program  policies  in  PSRO  Transmittal  Number  63. 

The  demonstration  projects  are  attempting  to  examine  and 
evaluate  different  methods  of  review.  All  discharges  and  all 
institutions  are  not  necessarily  reviewed.  The  projects  utilize 
various  methodologies,  including  prospective,  concurrent,  and 
retrospective  review.  About  half  of  the  demonstration  projects  have 
fully  or  almost  completely  implemented  long  term  care  review.  Four 
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have  implemented  review  in  about  half  of  the  nursing  homes  in  the 
area,  and  four  have  implemented  review  in  fewer  than  a  quarter  of 
the  nursing  homes. 

No  impact  data  are  yet  available  on  the  long  term  care  review 
program.  Such  data  may  be  avaialble  when  the  demonstrations  are 
reviewed  afer  two  years  of  operation  in  late  1978  and  when  the 
PSROs  performing  binding  review  have  had  more  experience.  It  is 
expected  that  approximately  five  additional  PSROs  will  begin  long 
term  care  review  in  fiscal  year  1979. 

Long  term  care  review  is  a  prerequisite  for  conversion  to  a 
fully  designated  status.  However,  due  to  a  limitation  of  Federal 
funds,  no  additional  PSROs  have  been  approved  for  long  term  care 
review.  This,  therefore,  has  impeded  PSROs  from  meeting  eligibility 
requirements  for  full  designation. 

7.3.5.2    Ambulatory  Care  Review 

The  Medicare-Medicaid  Anti-Fraud  and  Abuse  Amendments  (P.L. 
95-142)  changed  requirements  for  ambulatory  care  review.  In  the 
original  PSRO  law  (P.L.  92-603),  PSROs  had  the  option  of  requesting 
approval  to  review  non-institutional  care;  the  amendments  now 
require  PSROs  to  begin  review  of  ambulatory  care  not  later  than  two 
years  after  achieving  full  designation  status.  This  review  may, 
however,  begin  earlier  upon  application  to,  and  approval  by,  the 
Secretary. 

Since  there  is  no  generally  accepted  or  ideal  method  of  ambula- 
tory care  review,  program  policy  has  been  to  study  various  systems 
and  develop  a  strong  knowledge  base  before  significant  involvement 
by  PSROs.  In  October  1976,  DHEW  funded  five  PSROs  to  perform 
two-year  ambulatory  care  review  demonstration  projects.  These 
projects  were  intended  to  demonstrate  and  assess  the  effectiveness 
of  various  ambulatory  care  review  methodologies  and  thereby  assist 
in  the  overall  program  for  developing  policies  and  guidelines  in 
this  area.  Briefly,  the  five  demonstrations  have  the  following 
basic  features: 

( 1 )  The  Bronx  PSRO  uses  the  hospital  as  a  base  for  ambulatory 
care  review  in  a  high  volume  urbanized  setting.  Various 
diagnoses  were  selected  based  on  their  high  readmission 
rates.  The  ambulatory  care  received  between  the  initial 
hospitalization  and  the  readmission  was  analyzed  and, 
where  problems  were  encountered,  the  physician  was  con- 
tacted with  the  objective  of  modifying  patterns  of  care. 

(2)  The  Central  Massachusetts  PSRO  uses  a  hospital-based 
approach  which  centers  on  the  idea  that  hospitalization  for 
a  disease  for  which  the  admission  is  theoretically  prevent- 
able represents  a  poor  outcome  of  ambulatory  care.  The 
ambulatory  care  prior  to  such  hospitalizations  is  analyzed 
with  feedback  to  the  physicians. 
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(3)  The  South  Central  Pennsylvania  PSRO  is  integrating  a 
Blue  Shield  claim-based  review  system  and  an  in-depth 
quality  review  process.  Topics  chosen  by  the  PSRO  are 
screened  using  the  Blue  Shield  system  and  in-depth  phy- 
sician office  audits  are  performed  in  possible  problem 
areas  with  intervention  where  indicated. 

(4)  The  New  Mexico  PSRO  uses  a  claim-based  system  to  review 
patterns  of  ambulatory  care  practice.  Patterns  of  care 
are  developed  from  claims  data,  problems  identified  by 
physicians,  and  an  educational  intervention  begun  to 
modify  practice  patterns. 

(5)  The  Multnomah  Foundation  for  Medical  Care  is  studying 
different  methods  of  auditing.  Critical  and  screening 
criteria  for  use  in  MCE  studies  are  being  developed  and 
tested.  Also,  the  relative  attributes  of  traditional 
feedback  versus  a  concurrent  educational  feedback  tech- 
nique for  physicians  are  being  analyzed. 

Upon  completion  of  the  demonstration  projects,  an  evaluation 
report  is  planned,  and  the  report  will  be  used  by  the  program  to 
help  determine  the  future  involvement  of  PSRO  in  ambulatory  care 
review.  In  spring  1978,  New  York  County  Health  Services  Review 
Organization  (2NY11)  was  funded  to  review  care  provided  in  shared 
health  facilities.  Also,  an  additional  five  demonstration  projects 
may  be  funded  in  fiscal  year  1979. 

7.3.5.3    Ancillary  Service  Review 

The  review  of  ancillary  services  associated  with  certified 
admissions  is  expected  to  further  decrease  the  cost  of  hospitaliza- 
tion by  eliminating  inappropriate  services.  An  ancillary  service 
may  be  defined  as  any  service  offered  by  a  hospital  for  which  a 
charge  separate  from  the  routine  room  and  board  charge  is  incurred 
in  such  departments  as  laboratory,  radiology,  pharmacy,  occupational 
therapy  and  speech  therapy.  Within  these  departments,  literally 
hundreds  of  different  services  and  procedures  can  be  independently 
ordered. 

HSQB  has  funded  ten  demonstrations  for  ancillary  service 
review  in  fiscal  year  1978.  The  demonstrations  are  undertaking 
review  of  ancillary  services  as  a  part  of  their  regular  acute  care 
review,  some  concurrently  and  some  retrospectively.  Four  general 
models  of  review  are  (1)  retrospective  pre-payment  review  of  claims, 
(2)  concurrent  review  of  ancillary  services,  (3)  prior  review  of 
specific  services,  and  (4)  special  retrospective  studies  (frequently 
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in  the  context  of  MCEs  for  a  given  diagnostic  category  or  ancillary 
procedure)*  The  methods  used  to  identify  major  inappropriate 
utilization  of  ancillary  services  are  concurrent  review,  profile 
analysis,  and  MCEs. 

A  report  will  be  prepared  discussing  the  various  experiences 
in  order  to  assist  other  PSROs  in  implementing  ancillary  services 
review  and  to  assist  HSQB  in  developing  its  general  policies. 
Perhaps  the  most  valuable  output  of  the  ancillary  services  review 
demonstration  sites  would  be  testing  of  methods  for  selecting 
a  review  focus  and  developing  a  common  set  of  selective  criteria. 
Increased  emphasis  is  expected  to  be  placed  on  ancillary  services 
review  in  FY  1979. 
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8.     DIRECTIONS  FOR  FUTURE  EVALUATION 


A  number  of  specific  evaluation  studies,  scheduled  for  future 
completion,  are  summarized  in  this  section.  The  studies  will 
involve  (a)  analyses  yet  to  be  designed,  (b)  contracts  currently 
funded  and  underway,  and  (c)  evaluations  of  demonstrations  conducted 
in  the  field.  The  studies,  which  include  additional  refinements 
and  innovative  approaches  to  measuring  the  comprehensive  impact  of 
all  program  components,  are  divided  into  five  basic  areas. 

a.  Implementation  studies 

b.  Quality  of  care 

c.  Program  statistics,  implementation  and  development 

d.  Program  expenditures 

e.  Demonstrations  of  alternatives  to  PSROs. 

Brief  summaries  of  the  studies  currently  underway  or  planned 
are  provided  below: 

Title  of  Study  Summary  

A.     Utilization  Studies 

(1)  Medicare  100%  Medicare  claims  file  data 

Claim  File  will  be  used  to  review  days 

of  care  per  1000  benefici- 
aries, average  length 
of  stay,  discharge  rates, 
incidence  of  short  stays, 
incidence  of  long  stays,  and 
other  aspects  of  beneficiary 
uti lization. 


(2)  Medicare  20%  Examination  of  changes  in 

Research  File  utilization  due  to  PSRO 

for  potentially  abused 
diagnoses  and  inefficacious 
surgical  procedures. 


(3)  PSRO  Hospital  Dis- 
charge Data  Set 
(PHDDS) 


Involves  three  components: 

a.  Examination  of  changes 
in  variation  about 
observed  ALOS  for  both 
Medicare  and  Medicaid. 

b.  Examination  of  correlation 
between  changes  in  days 

of  care  per  1000  bene- 
ficiaries, and  noncer- 
tified  days. 

c.  Staging  analyses  for 
both  Medicare  and  Medi- 
caid. 
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Title  of  Study 


Summary 


B.    Quality  of  Care 
( 1 )  MCE  Study 


MCE  implementation  progress 
will  be  reported.  Data 
from  MCE  audits  and  reaudits 
will  be  examined  to  determine 
impacts  on  health  outcomes 
and  hospital  expenditures  for 
a  small  sample  of  diagnoses. 
Potential  benefit-cost  ratios 
of  selected  studies  will  be 
estimated. 


(2)  MCE  Topic  Selection 
and  Criteria  Setting 


Topic  selection,  criteria 
setting,  and  the  justifi- 
cation of  variations  found 
through  MCEs  will  be  examined 
using  in-house  MCE  reports 
and  data  from  surveys  and 
interviews  in  the  field. 


(3)  Unnecessary  Days 
Pilot  Study 


Analysis  of  Massachusetts 
PSRO  inappropriate  hospitali- 
zation study. 


(4)  Unnecessary  Surgery 


Pilot  study  in  5  PSROs 
involving  monitoring 
suspect  surgery  with  use  of 
a  variety  of  sanctions. 
Examination  of  changes  in 
incidence  of  monitored 
surgery  and  all  surgery. 


(5)  Profile  Analysis 


Study  on  implementation  of 
profiling  and  its  impact  on 
other  PSRO  and  hospital 
review  processes. 
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Title  of  Study   Summary 

C.    Program  Statistics, 
Implementation  and 
Development 


( 1 )  Update  of  Program         Provides  descriptive  statis- 
Status  Report  tics  on  program  implementa- 

tion, status,  and  progress. 


(2)  Objective  Setting        This  study  will  examine 
and  Self -Evaluation     the  correspondence  between 

HSQB  objectives,  local 
objectives,  and  local  PSRO 
performance.  In  addition, 
the  use  of  HCFA  Medicare, 
Medicaid,  and  PMIS  data  to 
supplement  PSRO  self-eval- 
uations will  be  explored. 


(3)  Developmental  study 
for  baseline  data, 
data  base  develop- 
ment planning  of 
PSRO  Demonstration 
Projects 


Existing  models  for  ambula- 
tory review,  long  term  care 
review,  and  ancillary  ser- 
vices review  will  be 
examined  to  determine 
data  needs  for  future 
eva luations .  Selected 
PSROs  will  participate 
in  a  project  to  assemble 
pre-review  data  bases  for 
future  evaluation  studies 
in  these  areas. 


(4)  Organizational 

Effectiveness  Study 


This  study  will  attempt 
to  develop  a  broad  model 
of  PSRO  performance 
to  assess  those  program 
parameters  most  closely 
associated  with  positive 
impact. 
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Title  of  Study  Summary  

D.    Program  Expenditures 

( 1 )  Update  of  Status  Report  on  gross  variations 

Report  in  PSRO  budgets.  Specific 

examination  of  expenditures 
for  data  systems  and  data 
base  development. 


(2)  Cost  Reductions  Variations  in  the  methods 

Through  Focused  used  in  focused  review  will 

Review  be  examined  to  determine 

those  methods  of  general 
applicability  which  can 
produce  the  greatest  savings. 


E.    Demonstrations  of 

Alternatives  to  PSRO 


( 1 )  Evaluation  of  New 
York  Demonstration 


Comparison  of  relative  effec- 
tiveness of  New  York  PSROs 
versus  traditional  review  by 
state  Medicaid  agency. 


(2)  Evaluation  of 
Oklahoma  Demon- 
stration 


Assessment  of  use  of  computer- 
ized screens  followed  by  con- 
current review  focused  upon 
physicians  suspected  of  high 
utilization. 


(3)  Second  Opinion 
Surgery  Demon- 
stration 


Assessment  of  potential  of 
second  surgical  opinion  for 
reducing  unnecessary  surgery. 
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Appendix  I.     PSRO  Implementation  Status 


Column  Column  Title 

(1)  Area  funding  status  (P-planning,  C-conditional,  U-unf unded) ,  7/1/78 

(2)  Date  of  conditional  contract  award 

(3)  Percent  of  hospitals  implemented,  6/30/77 

(4)  Percent  of  implemented  hospitals  under  fully  delegated  review,  6/30/77 

(5)  Percent  of  implemented  hospitals  under  partially  delegated  review,  6/30/77 

(6)  Percent  of  implemented  hospitals  under  non-delegated  review,  6/30/77 

(7)  Percent  of  hospitals  implemented,  6/30/78 

(8)  Percent  of  implemented  hospitals  under  fully  delegated  review,  6/30/78 

(9)  Percent  of  implemented  hospitals  under  partially  delegated  review,  6/30/78 

(10)  Percent  of  implemented  hospitals  under  non-delegated  review,  6/30/78 

(11)  Percent  of  reviewed  Federal  discharges  under  fully  delegated  review, 

July-September  1977 

(12)  Percent  of  reviewed  Federal  discharges  under  partially  delegated  review, 

July-September  1977 

(13)  Percent  of  reviewed  Federal  discharges  under  non-delegated  review, 

July-September  1977 
(1A)      Percent  of  total  discharges  made  up  by  Federal  discharges,  1977 

(15)  Percent  of  reviewed  Federal  admissions  granted  certification, 

July-September  1977 

(16)  Percent  of  certified  Federal  admissions  reviewed  for  continued  stay, 

July-September  1977 

(17)  Percent  of  eligible  physicians  who  are  members  of  the  PSRO,  1977 

(18)  Long-term  care  binding  review  site 

(19)  Long-term  care  review  demonstration  site 

(20)  Ambulatory  care  review  demonstration  site 

(21)  Focused  review  demonstration  site 

(22)  Ancillary  services  review  demonstration  site 

(23)  Surgery  evaluation  project  site 

(24)  Self-evaluation  reported 

(25)  EMCRO  site  or  other  previous  review 

Sources 

Columns    1-2  :  HSQB  Contracts  Office 

Columns    3-6  :  HSQB  Implementation  Status  Report,  2d  Quarter  1977 

Columns    7-10:  HSQB  Implementation  Status  Report,  2d  Quarter  1978 

Columns  11-17:  BQA  Form  121  Summaries,  HSQB  Division  of  Data  Planning 

Columns  18-25:  HSQB,  Division  of  Program  Operations 

Notes 

NA  (in  columns  3-10)  -  Not  Applicable  (PSRO  conditional  for  less  than  1  month). 
NA  (elsewhere)  -  Not  Available. 

*  (in  column  15)  includes  one  uncertified  admission  but  calculation  rounds  off 
to  100.00  percent. 

In  columns  11-13,  where  there  are  discharges  under  an  indicated  delegation  status 
but  not  enough  to  round  off  to  1  percent,  1  percent  is  entered  to  indicate  the 
presence  of  that  review. 

Health  Care  Financing  Administration 
Office  of  Policy,  Planning,  and  Research 
November  1978 
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Appendix  I.    PSRO  Implementation  Status 


PSRO 

number 

(1) 

(2) 

(3) 

(4) 

(5) 

(6) 

(7) 

(8) 

(9) 

(10) 

(11) 

(12) 

(13) 

(14) 

(15) 

(16) 

(17) 

(18) 

(19) 

(20)  (21) 

(22) 

(23) 

(24) 

(25) 

4AL01 

c 

6/1/76 

47 

93 

0 

7 

100 

68 

0 

32 

97 

0 

3 

34 

99.98 

55 

70 

OAK01 

c 

6/30/76 

28 

100 

0 

0 

71 

76 

0 

24 

83 

12 

5 

48 

100.00 

35 

53 

9AZ01 

p 

- 

NA 

NA 

9AZ02 

c 

9/30/77 

NA 

NA 

MA 

HA 

NA 

40 

6AR01 

c 

6/27/75 

91 

99 

0 

1 

90 

99 

0 

1 

99 

0 

1 

40 

98.41 

37 

67 

X 

9CA01 

c 

6/26/75 

100 

35 

61 

4 

100 

38 

58 

4 

47 

61 

X 

9CA02 

c 

6/30/78 

NA 

NA 

NA 

HA 

NA 

HA 

9CA03 

c 

6/26/75 

100 

58 

34 

8 

100 

42 

42 

16 

45 

51 

4 

40 

100.00 

46 

62 

9CA04 

c 

4/1/75 

100 

0 

0 

100 

100 

9 

0 

91 

0 

0 

100 

41 

100.00 

39 

71 

X 

X 

9CA05 

c 

9/1/76 

89 

100 

0 

0 

95 

100 

0 

0 

100 

0 

0 

51 

99.49 

73 

57 

X 

9CA06 

p 

- 

37 

42 

9CA07 

p 

- 

NA 

NA 

9CA08 

c 

6/29/74 

100 

0 

62 

38 

100 

0 

57 

43 

0 

82 

18 

56 

99.97 

36 

69 

X 

X 

9CA09 

c 

6/29/76 

14 

100 

0 

0 

100 

85 

0 

15 

100 

0 

0 

38 

99.76 

75 

60 

X 

9CA10 

c 

9/30/77 

100 

65 

29 

6 

NA 

NA 

9CA11 

p 

- 

48 

46 

9CAL2 

c 

6/26/75 

100 

36 

46 

18 

100 

36 

46 

18 

48 

48 

4 

50 

100.00 

58 

68 

9CA13 

c 

6/30/78 

NA 

NA 

HA 

NA 

57 

83 

9CA14 

c 

5/27/76 

100 

62 

0 

38 

100 

62 

7 

31 

90 

0 

10 

46 

100.00 

47 

67 

X 

9CA15 

c 

3/30/78 

NA 

NA 

NA 

HA 

44 

45 

9CA16 

c 

3/29/77 

NA 

NA 

NA 

NA 

93 

92 

8 

0 

100 

0 

0 

50 

100.00 

43 

70 

9CA17 

c 

6/30/76 

36 

100 

0 

0 

100 

82 

0 

16 

100 

0 

0 

39 

99.55 

50 

87 

X 

9CA18 

c 

12/30/77 

44 

17 

0 

63 

53 

28 

9CA19 

p 

- 

NA 

NA 

9CA20 

c 

9/21/76 

28 

100 

0 

0 

89 

100 

0 

0 

100 

0 

0 

41 

96.32 

64 

55 

9CA21 

c 

9/30/77 

21 

83 

0 

17 

38 

26 

9CA22 

c 

6/23/76 

23 

100 

0 

0 

83 

100 

0 

0 

100 

0 

0 

45 

99.97 

65 

100 

X 

9CA23 

c 

1/5/77 

14 

100 

0 

0 

100 

42 

0 

58 

100 

0 

0 

41 

99.56 

62 

48 

9CA24 

c 

6/30/75 

39 

100 

0 

0 

93 

100 

0 

0 

100 

0 

0 

50 

97.01 

73 

51 

9CA25 

c 

3/30/78 

NA 

NA 

NA 

NA 

62 

HA 

9CA26 

p 

- 

NA 

NA 

9CA27 

c 

6/26/75 

93 

86 

0 

14 

100 

87 

0 

13 

95 

2 

50 

82.28 

48 

78 

X 

X 

9CA28 

p 

- 

NA 

HA 

BC001 

c 

6/28/74 

89 

5 

0 

95 

100 

5 

0 

95 

13 

0 

87 

29 

100.00 

53 

60 

X 

X 

X 

1CT01 

c 

6/30/75 

100 

100 

0 

0 

100 

100 

0 

0 

100 

0 

0 

32 

99.40 

55 

88 

X 

X 

1CT02 

c 

6/28/75 

92 

91 

0 

9 

100 

92 

0 

8 

HA 

63 

X 

X 

1CT03 

c 

6/27/75 

89 

88 

0 

12 

100 

89 

0 

11 

99 

0 

1 

33 

99.96 

61 

72 

X 

1CT04 

c 

6/30/75 

100 

100 

0 

0 

100 

100 

0 

0 

27 

75 

X 

3DE01 

c 

6/1/76 

71 

60 

20 

20 

100 

72 

14 

14 

73 

27 

0 

34 

99.95 

60 

77 

X 

3DC01 

c 

6/28/75 

100 

24 

38 

38 

100 

20 

33 

47 

18 

40 

42 

90 

100.00 

46 

44 

X 

X 

X 

I 

4FL01 

p 

- 

32 

HA 

4FL02 

c 

3/30/78 

NA 

NA 

HA 

NA 

33 

50 

4FL03 

c 

3/30/77 

HA 

NA 

NA 

NA 

100 

100 

0 

0 

100 

0 

0 

29 

86.60 

69 

HA 

4FL04 

p 

- 

HA 

HA 

4PL05 

c 

6/30/78 

NA 

NA 

HA 

NA 

36 

45 

4FL06 

p 

_ 

33 

64 

4FL07 

p 

- 

HA 

HA 

4FL08 

p 

- 

NA 

NA 

4FL09 

p 

- 

59 

17 

4FL10 

p 

- 

39 

48 

4FL11 

c 

6/30/78 

NA 

NA 

NA 

NA 

NA 

36 

4F1.12 

c 

12/15/75 

100 

90 

0 

10 

100 

82 

5 

13 

99 

0 

1 

NA 

95.40 

60 

NA 

X 

4GA01 

p 

HA 

NA 

X 

9HI0I 

c 

6/23/76 

18 

100 

0 

0 

47 

88 

12 

0 

100 

0 

0 

40 

99.22 

55 

58 

X 

OI0O1 

c 

6/26/75 

100 

4 

0 

96 

100 

8 

0 

92 

NA 

41 

5TL01 

c 

3/30/78 

NA 

NA 

HA 

NA 

29 

52 

X 

5IL02 

c 

3/30/78 

NA 

NA 

HA 

NA 

23 

31 

z 

5IL03 

c 

9/30/76 

17 

93 

0 

7 

100 

93 

0 

7 

92 

0 

8 

41 

94.45 

62 

41 

X 

SIL04 

c 

6/26/75 

100 

67 

0 

33 

100 

83 

0 

17 

80 

0 

2G 

28 

100.00 

52 

100 

X 

X 

5IL05 

c 

3/30/78 

72 

83 

0 

17 

28 

47 

X 

5IL06 

c 

9/30/77 

NA 

NA 

NA 

NA 

33 

43 

X 

5IL07 

u 

HA 

X 

5TT.0B 

c 

3/30/78 

57 

100 

0 

0 

44 

49 

X 

5i:wi 

c 

9/30/77 

NA 

NA 

HA 

NA 

22 

56 

5IS02 

c 

3/30/78 

NA 

NA 

HA 

NA 

NA 

30 
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Appendix  1.     PSRO  Implementation  Status  (Continued) 


PSRO 

number 

/ 1  \ 

(2) 

(3) 

(4) 

(5) 

(6) 

(7) 

(g) 

(9) 

(10) 

(11) 

(12) 

(13) 

(14) 

(15) 

(16) 

(17) 

(18) 

5IN03 

C 

6/30/78 

NA 

NA 

NA 

NA 

26 

35 

5TN04 

c 

3/30/78 

NA 

NA 

NA 

NA 

29 

26 

5IN05 

c 

9/30/77 

NA 

NA 

NA 

NA 

25 

NA 

SIN06 

c 

3/30/78 

NA 

NA 

NA 

NA 

NA 

NA 

51  HO  7 

c 

6/30/78 

NA 

NA 

NA 

NA 

29 

69 

7IA01 

c 

12/15/75 

24 

100 

0 

0 

63 

100 

0 

0 

100 

0 

0 

NA 

96.32 

48 

56 

7KS01 

c 

9/30/77 

13 

100 

0 

0 

NA 

NA 

4KY01 

c 

9/15/76 

36 

85 

0 

15 

98 

71 

0 

29 

87 

0 

13 

34 

94.78 

53 

56 

6LA01 

p 

33 

11 

6LAQ2 

p 

29 

25 

6LA03 

r 

- 

43 

13 

6LAM 

t 

37 

15 

1ME01 

c 

6/27/75 

85 

98 

0 

2 

93 

94 

0 

6 

97 

0 

3 

42 

98.13 

42 

50 

3MD01 

c 

6/30/76 

100 

100 

0 

0 

100 

100 

0 

0 

100 

0 

0 

34 

ICO. 00 

53 

56 

X 

3HD02 

c 

6/27/75 

100 

94 

0 

6 

100 

94 

0 

6 

95 

5 

0 

38 

99.98 

98 

71 

3MD03 

c 

6/27/75 

100 

50 

0 

50 

100 

50 

0 

50 

49 

0 

51 

23 

99.35 

62 

1 

X 

3MKM 

c 

6/29/7* 

100 

100 

0 

0 

100 

17 

0 

83 

0 

0 

100 

29 

99.95 

70 

47 

X 

3MD05 

c 

6/27/75 

100 

100 

0 

0 

100 

100 

0 

0 

100 

0 

0 

26 

100.00 

70 

75 

X 

3MD06 

c 

6/27/75 

100 

40 

40 

20 

100 

40 

40 

20 

41 

47 

12 

27 

99.91 

64 

70 

X 

3MD07 

c 

6/27/75 

100 

17 

83 

0 

100 

17 

83 

0 

23 

77 

0 

42 

100.00 

41 

66 

X 

1MA01 

c 

6/27/75 

100 

100 

0 

0 

100 

100 

0 

0 

100 

0 

0 

40 

99.67 

45 

54 

X 

1MA02 

c 

6/1/76 

81 

92 

0 

8 

100 

94 

0 

6 

95 

0 

5 

38 

98.62 

55 

82 

1MA03 

c 

6/29/74 

100 

88 

12 

0 

100 

88 

12 

0 

100 

0 

0 

33 

99.65 

49 

NA 

1MA04 

c 

6/28/74 

97 

61 

0 

39 

97 

66 

7 

27 

34 

0 

66 

37 

100.00* 

39 

58 

IMAOS 

c 

6/30/75 

93 

100 

0 

0 

100 

93 

7 

0 

100 

0 

0 

39 

99.90 

56 

56 

X 

SMI  01 

c 

6/30/75 

100 

5 

0 

95 

100 

5 

0 

95 

40 

74 

SMI  02 

r 

NA 

NA 

5MI03 

c 

3/30/78 

NA 

NA 

NA 

NA 

36 

33 

SMI  04 

c 

6/30/78 

NA 

NA 

NA 

NA 

35 

65 

smos 

c 

6/27/75 

100 

22 

0 

78 

100 

22 

0 

78 

37 

60 

5MI 06 

c 

6/30/78 

NA 

NA 

NA 

NA 

33 

51 

smo7 

c 

3/30/78 

NA 

NA 

NA 

NA 

31 

42 

5MI08 

c 

9/30/77 

NA 

NA 

NA 

NA 

19 

44 

5MI09 

p 

27 

HA 

staio 

p 

39 

52 

SMN01 

c 

6/28/74 

100 

100 

0 

0 

100 

100 

0 

0 

100 

0 

0 

33 

99.19 

49 

37 

SMK02 

c 

6/30/76 

NA 

NA 

NA 

NA 

26 

100 

0 

0 

100 

0 

0 

35 

97.26 

50 

52 

4MS01 

c 

6/29/74 

93 

4b 

47 

7 

100 

46 

46 

8 

49 

47 

4 

38 

99.96 

41 

70 

X 

7MD01 

c 

9/30/76 

23 

100 

0 

0 

61 

100 

0 

0 

100 

0 

0 

36 

99.65 

50 

49 

7M0O2 

c 

6/1/76 

100 

84 

3 

13 

100 

85 

0 

15 

81 

0 

19 

41 

99.88 

54 

54 

7M0O3 

c 

6/27/75 

49 

100 

0 

0 

83 

100 

0 

0 

100 

0 

0 

31 

99.96 

63 

38 

X 

7HD04 

c 

7/22/76 

80 

100 

0 

0 

96 

91 

0 

9 

97 

0 

3 

40 

99.99 

46 

ou 

7HD05 

c 

6/2/76 

95 

95 

0 

5 

100 

86 

10 

4 

49 

73 

SMT01 

c 

5/8/75 

99 

5 

2 

92 

99 

5 

2 

92 

17 

3 

80 

NA 

100.00 

38 

43 

X 

7MB01 

0 

NA 

9BTO1 

c 

6/30/76 

96 

74 

13 

13 

100 

78 

13 

9 

98 

1 

1 

39 

99.99 

51 

55 

X 

1NH01 

c 

6/27/75 

96 

93 

0 

7 

100 

89 

0 

11 

NA 

X 

2!U01 

c 

5/28/76 

100 

100 

0 

0 

100 

100 

0 

0 

26 

65 

21U02 

c 

6/27/75 

100 

100 

0 

0 

100 

14 

86 

0 

100 

0 

0 

36 

99.79 

59 

61 

2HJ03 

c 

3/30/78 

NA 

NA 

NA 

NA 

33 

44 

2BJ04 

c 

7/21/76 

34 

100 

0 

0 

82 

93 

7 

0 

100 

0 

0 

38 

99.93 

69 

69 

2HJ05 

c 

3/30/78 

NA 

NA 

HA 

NA 

43 

61 

2NJ06 

c 

3/30/78 

NA 

NA 

HA 

NA 

31 

52 

2!U07 

c 

6/30/78 

NA 

NA 

HA 

NA 

31 

35 

2HJ08 

c 

9/21/76 

NA 

NA 

NA 

NA 

33 

100 

0 

0 

37 

47 

6HM01 

c 

11/1/74 

86 

78 

0 

22 

100 

57 

0 

43 

28 

100.00 

43 

44 

X 

2HT01 

c 

6/27/75 

61 

96 

0 

4 

84 

90 

0 

10 

0 

0 

100 

23 

45 

2NV02 

c 

6/27/75 

100 

95 

0 

5 

100 

100 

0 

0 

95 

0 

5 

33 

99.94 

47 

59 

2OT03 

c 

9/21/76 

NA 

NA 

NA 

NA 

41 

91 

0 

9 

100 

0 

0 

36 

93.49 

57 

59 

2HY04 

c 

0/30/ 76 

15 

100 

0 

0 

46 

100 

0 

0 

100 

0 

0 

39 

96.90 

40 

83 

2NT05 

c 

6/28/75 

81 

100 

0 

0 

87 

100 

0 

0 

100 

0 

0 

39 

99.41 

50 

72 

2NT06 

c 

3/30/78 

NA 

NA 

HA 

NA 

41 

74 

2NY07 

c 

3/30/78 

NA 

NA 

HA 

NA 

20 

44 

2  NY  08 

c 

3/30/78 

NA 

NA 

HA 

NA 

39 

99.83 

60 

34 

JHYM 

c 

6/27/75 

100 

100 

0 

0 

100 

100 

0 

0 

100 

0 

0 

39 

45 

I  X 
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Appendix  1.    I'SRO  laplcaentatlon  Status  (Continued) 


PS  HO 

nusnbor 

<1) 

(2) 

(3) 

(4) 

(5) 

(6) 

(7) 

(8) 

(9) 

(10) 

(11) 

(12) 

(13) 

(14) 

(15) 

(16) 

(17) 

2NY10 

C 

6/27/75 

100 

100 

0 

0 

100 

100 

0 

0 

100 

0 

0 

32 

99.84 

54 

73 

JNY'l 1 

c 

t>/2  7/75 

58 

72 

0 

28 

83 

60 

0 

40 

7.2 

9 

19 

NA 

99.20 

51 

42 

-SY12 

c 

,»/3Q/76 

25 

100 

0 

0 

100 

100 

0 

0 

100 

0 

0 

NA 

54 

74 

2KY13 

c 

6/27/75 

66 

100 

0 

0 

96 

100 

0 

0 

100 

0 

0 

S3 

98.41 

48 

49 

2  KYI  A 

c 

1/31/77 

19 

100 

0 

0 

86 

83 

6 

11 

100 

0 

0 

45 

95.40 

62 

49 

2NY15 

c 

6/27/75 

100 

100 

0 

0 

100 

100 

0 

0 

100 

0 

0 

31 

99.81 

61 

51 

2  NY  16 

c 

6/27/75 

92 

100 

0 

0 

100 

85 

0 

IS 

93 

7 

0 

69 

54 

46 

2NY17 

c 

6/30/78 

NA 

NA 

RA 

HA 

NA 

RA 

4KC01 

c 

3/30/78 

HA 

NA 

HA 

HA 

32 

55 

4KO02 

c 

3/1/77 

MA 

HA 

NA 

NA 

91 

25 

0 

75 

0 

0 

100 

39 

100.00 

HA 

73 

4KC03 

c 

3/30/78 

NA 

HA 

RA 

RA 

25 

57 

4NC04 

c 

3/30/78 

NA 

NA 

RA 

HA 

NA 

NA 

4NC05 

c 

3/30/78 

HA 

NA 

HA 

RA 

27 

70 

4WC06 

c 

3/30/78 

NA 

RA 

RA 

HA 

32 

46 

4NC07 

c 

9/30/77 

NA 

HA 

RA 

HA 

HA 

RA 

4WC08 

c 

3/30/78 

NA 

RA 

RA 

RA 

34 

53 

8TO01 

c 

9/22/76 

79 

44 

54 

2 

100 

31 

60 

9 

72 

27 

1 

38 

100.00 

52 

75 

5OH01 

c 

6/27/75 

100 

92 

0 

8 

100 

92 

0 

6 

96 

0 

4 

HA 

99.51 

62 

61 

50802 

c 

9/9/76 

87 

100 

0 

0 

100 

93 

0 

7 

98 

0 

2 

32 

99.88 

65 

54 

SOH03 

c 

3/30/78 

NA 

HA 

RA 

RA 

30 

62 

50H34 

c 

6/26/75 

100 

89 

0 

11 

100 

84 

0 

16' 

90 

0 

10 

28 

99.93 

93 

76 

50H05 

c 

3/30/78 

NA 

HA 

HA 

RA 

HA 

RA 

S0HO6 

c 

9/30/77 

71 

100 

0 

0 

RA 

66 

50H37 

c 

6/30/78 

NA 

HA 

HA 

NA 

31 

27 

30KO8 

c 

6/30/78 

NA 

NA 

M 

NA 

32 

RA 

50H09 

c 

3/30/78 

NA 

NA 

RA 

HA 

37 

83 

S0H10 

c 

9/1/76 

100 

76 

0 

24 

100 

76 

0 

24 

86 

0 

14 

31 

100.00 

57 

31 

50H11 

c 

3/30/78 

NA 

HA 

RA 

HA 

25 

33 

50H12 

c 

6/30/76 

45 

100 

0 

0 

100 

77 

0 

23 

93 

0 

7 

HA 

99*96 

Alt 

RA 

50H13 

p 

RA 

NA 

60K01 

c 

3/30/78 

91 

100 

0 

0 

HA 

RA 

O0R01 

c 

6/28/76 

100 

43 

50 

7 

100 

54 

31 

IS 

47 

35 

18 

NA 

101).  03 

48 

63 

OCR  32 

c 

6/30/75 

79 

98 

0 

2 

89 

98 

0 

2 

99 

0 

1 

37 

99.99 

38 

65 

3PA01 

c 

12/29/77 

NA 

HA 

HA 

HA 

35 

62 

3PA02 

c 

6/3/76 

63 

100 

0 

0 

79 

100 

0 

0 

100 

0 

0 

39 

99.58 

54 

69 

3PA03 

c 

3/30/78 

NA 

NA 

RA 

RA 

40 

32 

3FA04 

c 

6/1/76 

18 

100 

0 

0 

91 

90 

10 

0 

100 

0 

0 

34 

99.83 

66 

70 

3PA05 

c 

9/30/77 

50 

100 

0 

0 

NA 

HA 

3PA06 

c 

6/27/75 

100 

90 

7 

3 

100 

93 

0 

7 

90 

0 

10 

33 

99.89 

61 

40 

3PA07 

c 

6/27/75 

100 

83 

0 

17 

100 

83 

0 

17 

83 

1 

16 

41 

99.63 

52 

56 

3PA08 

c 

5/30/76 

62 

100 

0 

0 

100 

100 

0 

0 

100 

0 

0 

35 

99.46 

65 

47 

3PAM 

c 

6/27/75 

100 

100 

0 

0 

100 

100 

0 

0 

100 

0 

0 

NA 

99.83 

53 

RA 

3PA10 

c 

6/30/78 

NA 

NA 

HA 

RA 

34 

24 

3PA11 

c 

6/27/75 

83 

100 

0 

0 

71 

100 

0 

C 

100 

0 

0 

33 

99.98 

53 

61 

3PA12 

c 

6/27/75 

59 

100 

0 

0 

100 

84 

0 

16 

100 

0 

0 

56 

100.00 

51 

54 

2PR01 

c 

9/30/77 

4 

100 

0 

0 

NA 

HA 

lKIOl 

c 

6/30/75 

60 

100 

0 

0 

100 

100 

0 

0 

43 

(4 

4SC01 

c 

6/27/75 

90 

45 

0 

55 

100 

27 

0 

73 

59 

0 

41 

29 

100.00 

SO 

70 

8SD01 

c 

6/4/76 

98 

61 

0 

39 

100 

52 

0 

48 

81 

0 

19 

33 

99.88 

38 

72 

4TN01 

c 

4/1/77 

NA 

NA 

NA 

HA 

56 

100 

0 

0 

28 

34 

4TN02 

c 

6/29/74 

92 

35 

32 

33 

100 

35 

31 

34 

42 

39 

19 

RA 

99.92 

66 

HA 

6TX01 

0 

HA 

8UT01 

c 

6/18/74 

100 

0 

0 

100 

100 

0 

0 

100 

0 

0 

100 

23 

100.00 

1 

moi 

c 

6/1/76 

28 

100 

0 

0 

78 

100 

0 

0 

100 

0 

0 

40 

97.66 

45 

76 

2VI01 

c 

6/30/78 

NA 

HA 

RA 

RA 

53 

HA 

3VA01 

c 

9/30/77 

44 

100 

0 

0 

34 

52 

3VA02 

c 

1/3/77 

8 

100 

0 

0 

100 

100 

0 

0 

72 

0 

2S 

HA 

93.40 

43 

HA 

3VA03 

c 

6/30/78 

NA 

HA 

RA 

HA 

56 

36 

3VA04 

c 

6/30/78 

NA 

NA 

HA 

HA 

33 

40 

3VA05 

c 

1/5/77 

50 

100 

0 

0 

95 

95 

0 

5 

100 

0 

0 

26 

99.38 

70 

60 

OWA01 

c 

6/30/75 

80 

100 

0 

0 

100 

95 

0 

5 

97 

1 

2 

26 

99.96 

43 

58 

3WV01 

c 

6/28/75 

43 

100 

0 

0 

100 

71 

0 

29 

100 

0 

3 

33 

94.46 

47 

57 

5WI01 

c 

6/30/73 

100 

95 

0 

5 

100 
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GLOSSARY  OF  ABBREVIATIONS  AND  ACRONYMS  USED  IN  THIS  VOLUME 


AAPSRO  American  Association  of  Professional  Standards  Review 

Organizations 

AC  Admission  Certification 

ACR  Ambulatory  Care  Review 

AFDC  Aid  to  Families  with  Dependent  Children 

ALOS  Average  Length  of  Stay 

APTD  Aid  to  the  Partially  and  Totally  Disabled 

ARF  Area  Resource  File 

ASR  Ancillary  Services  Review 

AUTOGRP  Autogroup  (See  Glossary  of  Terms,  Appendix  III) 

BC/BS  Blue  Cross/Blue  Shield 

BHI  Bureau  of  Health  Insurance  (Social  Security 

Administration ) 

BQA  Bureau  of  Quality  Assurance 

CME  Continuing  Medical  Education 

CPHA  Commission  on  Professional  and  Hospital  Activities 

CPT  Current  Procedural  Terminology 

CR  Concurrent  Review 

CSR  Continued  Stay  Review 

DDR  Division  of  Direct  Reimbursement  (Social  Security 

Administration) 

DHEW  Department  of  Health,  Education  and  Welfare 

DISC  Discharges  (per  1000  enrollees) 

DO  Doctor  of  Osteopathy 

DOC  Days  of  Care 

DRG  Diagnostically  Related  Group 

ECF  Extended  Care  Facility 

ELOS  Estimated  Length  of  Stay 

EMCRO  Experimental  Medical  Care  Review  Organization 

FI  Fiscal  Intermediary 

FMC  Foundation  for  Medical  Care 

FRM  Federal  Reports  Manual 

FY  Fiscal  Year 

GAO  General  Accounting  Office 

GLSC  Professional  Review  Organization  -  Genessee, 

Lapeer,  Shaiwasee  Counties,  Michigan 

HCFA  Health  Care  Financing  Administration 

HDS  Hospital  Discharge  Survey  (National  Center  for  Health 

Statistics ) 

H-ICDA  Hospital-International  Classification  of  Diseases-Adapted 

HMO  Health  Maintenance  Organization 

HSA  Health  Systems  Agency 

HSQB  Health  Standards  and  Quality  Bureau 

HUP  Hospital  Utilization  Project  (Western  Pennsylvania) 

ICDA-8  International  Classification  of  Diseases-Adapted  (8th 
Revision) 

ICF  Intermediate  Care  Facility 

IOM  Institute  of  Medicine  (National  Academy  of  Sciences) 

JCAH  Joint  Commission  on  the  Accreditation  of  Hospitals 

LOS  Length  of  Stay 
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GLOSSARY  OF  ABBREVIATIONS  AND  ACRONYMS  USED  IN  THIS  VOLUME  (Cont'd) 


LTC  Long  Term  Care 

MAP  Medical  Audit  Program 

MCE  Medical  Care  Evaluation  (Studies) 

MCH  Maternal  and  Child  Health  (Program) 

MD  Doctor  of  Medicine 

MEDPAR  Medicare  Provider  Analysis  and  Review 

MFI  Master  Facilities  Index 

MOU  Memorandum  of  Understanding 

NCHS  National  Center  for  Health  Statistics 

NCHSR&D  National  Center  for  Health  Services  Research  and 
Development 

NPSRC  National  Professional  Standards  Review  Council 

OASH  Office  of  the  Assistant  Secretary  for  Health 

OMB  Office  of  Management  and  the  Budget 

OPEL  Office  of  Planning,  Evaluation  and  Legislation 

OPPR  Office  of  Policy,  Planning  and  Research 

OPSR  Office  of  Professional  Standards  Review 

OQS  Office  of  Quality  and  Standards 

PAS  Professional  Activity  Study 

PHDDS  PSRO  Hospital  Discharge  Data  Set 

PHS  Public  Health  Service 

PIP  Periodic  Interim  Payment 

PIPSRO  Private  Initiative  in  PSRO 

PL  Public  Law 

PMIS  PSRO  Management  Information  System 

PSRO  Professional  Standards  Review  Organization 

SNF  Skilled  Nursing  Facility 

SPSRC  Statewide  Professional  Standards  Review  Council 

SRS  Social  and  Rehabilitative  Services 

SSA  Social  Security  Administration 

SSI  Suplemental  Security  Income 

TAP  Technical  Advisory  Panel 

TDOC  Total  Days  of  Care  (rate  per  1000  enrollees) 

UHDDS  Uniform  Hospital  Discharge  Data  Set 

UR  Utilization  Review 
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STATE  ABBREVIATIONS 

Alabama  •  •  •  • . .  . .  .AL  Nebraska  o  .NE 

Alaska*  •••..o.......... « AK  Nevada  •••••••••«......  *NV 

Arizona » ...... .........  *AZ  New  Hampshiire<  .......c  »NH 

Arkansas  ...............  c  AR  New  Jersey  ...*...«..««  .NJ 

California.  ........ ...o  .CA  New  Mexico.  ...........  <>NM 

Colorado •••.•••••••••*«. CO  New  York .coocc©©c©..ctc Ny 

Connecticut. ........... .CT  North  Carolina  ..<>«>.  e . .  .NC 

Delaware. .............. .DE  North  Dakota. ......... «ND 

Dist .  of  Columbia. ..... .DC  Ohio.  ..o«®©oo©o©.ooooo  cOH 

Florida.  .......*••.....  .FL  Oklahoma  ■  » •  •  •  ■  ■  -  *  «  ■ .  ■  *0K 

Georgia.  .GA  Oregon   .OR 

Hawaii ............. .....HI  Pennsylvania PA 

Idaho. ................. .ID  Puerto  Rxco* .....»...» .PR 

Illinois                            .IL  Rhode  Island. . ...... .RI 

XXlcLiLeUftcl  •  ©•••©••••©••••••  IN  SO\ltll    CclSTOX  JLllcl  oqooooqqo  SO 

XOWS  •  •  •••••••©••••••••••  XJi  SOV&^ll    DcllCOfccl  •  *•••••••«•<»  t§0 

Kansas. ............  • . . .  .KS  Tennessee* . .  ?  •  . . . » ©  * . »  .TN 

Kentucky. ...........ooo  .Ky  Texas  *.o... .00.00.0000  oT3£ 

Louisiana.               0.00.  ©X*A  [Jtahr » » o  <»««-•«••.••••••  •  »UT 

Maine .«.......•••..*••. .ME  Vermont  *.....»...«.... . VT 

Maryland.                          .MD  -:-v     Islands  - 

Massachusetts  .......a*.  oMA  Virginia.  ••••^»»ooooo->  .VA 

Michigan. .......o...... .MI  Washington. .....9.3.00  ©WA 

Minnesota o .MN  West  Virginia o ........ ,WV 

Mississippi • ........so. .MS  Wisconsin. ....»•■...»* »WI 

Missouri ..o.... ....... 9 .MO  Wyoming. ..........o.^o  >WY 

Montana  •••••••••«••••©». MT 
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GLOSSARY  OF  TERMS 


Admission  Certification  (AC)  -  One  segment  of  the  Concurrent  Review 
component  of  the  model  PSRO  review  system  in  which  Federal 
patients'  admissions  are  reviewed  for  medical  necessity  and  the 
appropriateness  of  the  level  of  care,  using  diagnosis-specific 
admission  criteria  and  standards,  plus  the  assignment  of  a 
certified  length  of  stay  using  diagnosis-specific  norms, 
criteria  and  standards. 

Admission  Rate  -  The  number  of  admissions  to  a  health  care  facil- 
ity or  group  of  facilities  as  a  factor  of  a  unit  of  popula- 
tion or  program  enrollees,  usually  per  1000  population  or 
enrollees. 

Ambulatory  Care  -  Health  care  services  not  provided  on  an  in- 
patient basis. 

Ancillary  Services  -  Those  health  care  services  provided  in  a 
health  care  facility  other  than  general  nursing  care,  such 
as  laboratory,  radiology,  physical  therapy,  respiratory  therapy, 
etc. 

Assessment  and  Monitoring  -  Those  procedures  performed  by  a  fis- 
cal intermediary  or  state  agency  to  evaluate  the  review  pro- 
cedures of  a  PSRO  or  hospital  and  to  verify  the  results  of 
a  sample  of  the  reviews. 

Autogroup  (AUTOGRP)  -  An  interactive  statistical  computer  system 
for  the  analysis  of  health  care  data,  specifically  used  to 
develop  diagnostically  related  groups  (DRGs)  for  comparative 
analyses.  AUTOGRP  was  developed  by  the  Center  for  the  Study 
of    Health    Services    at   Yale   University   with   Federal  support. 

Average  Length  of  Stay  (ALOS)  -  A  measure  of  the  number  of  days 
stay  averaged  over  all  patients;  calculated  by  dividing  the 
total  days  of  care  by  the  number  of  discharges. 

Baseline  data  -  Data  representing  the  pre-treatment  period(s) 
of  a  study. 

Benefit-cost  -  The  ratio  of  the  dollar  value  of  the  impact  of 
a  program  or  activity  to  the  costs  incurred  in  carrying  out 
the  program  or  activity. 

Break-even  -  The  state  whereby  the  benefit  or  return  from  a  pro- 
gram or  activity  equals  the  costs  incurred. 

Break-even  point  -  The  combination  of  inputs  which  bring  about 
an  equal  return  (output). 


JJJ-2 


GLOSSARY  OF  TERMS  (Continued) 

Case  mix  -  The  relative  distribution  of  discharges  of  different 
diagnoses  or  diagnostic  groups. 

Case  mix  severity  -  A  measure  of  the  relative  distribution  of 
discharges  of  different  diagnoses  measured  by  the  aggregate 
expected  days  of  care  (calculated  from  expected  lengths  of 
stay  attributed  to  the  diagnoses  of  actual  cases). 

Claims  Review  -  The  review  of  claims  for  reimbursement  by  a  third 
party  payor,  generally  for  eligibility  and  coverage  determina- 
tions, but  sometimes  including  medical  necessity. 

Certified  Days  -  Hospital  inpatient  days  of  care  that  are  certi- 
fied by  a  PSRO  for  payment  at  the  hospital  level. 

Concurrent  Review  (CR)  -  One  component  of  the  model  PSRO  review 
system  where  institutionalization  of  federal  patients  is 
reviewed  while  the  patient  is  in  the  facility.  CR  is  com- 
posed   of    Admission    Certification    and    Continued    Stay  Review. 

Conditional  PSRO  -  An  organization  that  having  demonstrated  its 
capability  to  perform  the  duties  of  fully  designated  PSRO, 
enlisted  the  membership  of  at  least  25%  of  the  physicians 
in  the  area,  developed  a  review  and  implementation  plan,  and 
met  other  PSRO  organizational  requirements,  has  been  awarded  a 
contract  to  implement  the  required  review  and  other  functions 
in  its  designated  area. 

Cost  Effectiveness  -  The  state  whereby  the  benefit  of  a  program 
or  activity  equals  or  exceeds  the  costs  incurred;  a  benefit- 
cost-  ratio  of  1  to  1  or  greater. 

Continued  Stay  Review  (CSR)  -  One  segment  of  the  Concurrent  Review 
component  of  the  model  PSRO  review  system  in  which  Federal 
patients  whose  admission  has  been  certified  by  the  Admission 
Certification  process  are  periodically  reviewed  to  assure 
the  need  for  continued  hospital  care  and  that  care  is  being 
provided  efficiently  and  economically  using  diagnosis-specific 
norms,  standards,  and  criteria. 

Criteria  -  Predetermined  elements  of  care  against  which  aspects 
of  the  quality  of  a  medical  service  may  be  compared;  developed 
by  professionals  relying  on  professional  expertise  and  on  the 
professional  literature. 

Current  Procedural  Terminology  (CPT)  -  A  surgical  procedure  coding 
system  developed  by  the  American  Medical  Association. 
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GLOSSARY  OF  TERMS  (Continued) 


Days  of  Care  (DOC)  -  A  measure  of  utilization  equal  to  the  sum 
of  the  number  of  days  stay  for  all  inpatients;  also  equal 
to  the  product  of  the  number  of  admissions  and  the  average 
length  of  stay.  The  day  of  admission  is  counted  as  a  day  of 
care;  the  day  of  discharge  is  not. 

Delegation  -  The  assignment  of  Concurrent  Review,  Medical  Care 
Evaluation  Study  and/or  Reconsideration  functions  to  a  hos- 
pital committee  of  physicians  that  has  demonstrated  a  capability 
and  desire  to  perform  those  functions. 

Discharge  abstract  service  -  A  data  service  in  which  participating 
provider  institutions  submit  discharge  abstracts  which  are 
then  processed  and  reformatted  to  provide  various  utilization 
profiles . 

Federal  patient  -  A  patient  whose  care  is  paid  for  in  whole  or 
in  part  under  the  Social  Security  Act  (Title  V,  XVIII,  or 
XIX) . 

Fixed  Costs  -  That  portion  of  total  costs  which  does  not  vary 
with  changes  in  quantity. 

Focused  Review  -  Review  that  concentrates  on  a  perceived  problem 
area  that  may  be  a  specific  diagnosis,  procedure,  provider, 
or  other   limited  scope  topic,    in  lieu  of  comprehensive  review. 

Fully  Designated  PSRO  -  A  PSRO  that  has  performed  satisfactorily 
as  a  conditional  PSRO,  is  performing  hospital  and  long-term 
care  review,  and  has  been  granted  full  PSRO  status  as  speci- 
fied in  the  law. 

Health  Maintenance  Organization  (HMO)  -  A  prepaid  group  practice 
plan. 

Health  System  Agency  (HSA)  -  A  regionally-based  planning  organiza- 
tion established  pursuant  to  P.L.  93-641  and  charged  to  plan 
for  the  health  needs  of  all  residents  of  its  health  service 
area,  promote  the  development  of  needed  health  resources, 
and  reduce  inefficiencies. 

Length  of  Stay  (LOS)  -  A  measure  of  utilization  equal  to  the  number 
of  days  a  patient  is  institutionalized;  calculated  by  subtract- 
ing the  admission  date  from  the  discharge  date. 

Medicaid  State  Agency  -  A  state  governmental  agency  that  administers 
the  state's  Medicaid  program  and  normally  acts  as  an  intermedi- 
ary for  the  payment  of  Medicaid  claims. 
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Medical  Care  Evaluation  (MCE)  Studies  -  One  component  of  the  model 
PSRO  review  system  whereby  a  medical  care  topic  area  is  selected 
and  studied  on  a  sample  basis,  usually  retrospectively,  to 
assure  the  quality  of  care  and  effective  administration  of 
health  care  services.  Generally  the  process  involves  choosing 
a  topic,  establishing  criteria  and  standards,  collecting  and 
analyzing  data  to  identify  deficiencies,  implementing  corrective 
action  and  restudying  the  topic. 

Medicare  fiscal  intermediary  -  An  organization  which  has  contracted 
with  DHEW  to  process  claims  for  reimbursement  of  medical  care 
under  Title  XVIII  of  the  Social  Security  Act  and  which  actually 
pays  such  claims  as  an  agent  for  the  Federal  government. 

Medicare  Part  A  -  Part  of  Title  XVIII  of  the  Social  Security  Act 
providing  hospital  insurance  benefits  to  the  aged  and  disabled 
covering    hospitalization    and    related    post-hospital  , services. 

Medicare  Part  B  -  Part  of  Title  XVIII  of  the  Social  Security  Act 
providing  voluntary  supplemental  medical  insurance  benefits 
to  the  aged  and  disabled  covering  physician  and  other  profes- 
sional   services,    outpatient    services,    home    health  services, 

etc. 

Norms    -   Numerical   or   statistical  measures   of  usual  observed  per- 

f  ormance . 

Objective  setting  -  Part  of  HSQB's  program  initiatives  whereby 
PSROs   set  performance  objectives   to  enhance  impact  and  control 

costs. 

Patient  day  -  A  unit  of  utilization  consisting  of  one  day  of  stay 

for  one  patient. 

Per  Diem  Charge  -  The  average  total  charge  for  a  day  of  care  (one 
patient  day),  including  both  accommodations  and  ancillary 
charges. 

Planning  PSRO  -  An  organization  which  has  demonstrated  the  poten- 
tial to  meet  the  requirements  for  conditional  PSRO  designation 
and  has  been  awarded  a  contract  for  financial  assistance  to 
develop  that  potential  by  developing  a  review  plan  and  imple- 
mentation plan,  recruiting  physician  members,  and  collecting 
baseline  data. 

Pre-admission  Certification  -  Performance  of  the  Admission  Certifi- 
cation process  on  elective  cases  prior  to  admission.  Pre- 
admission certification  may  be  performed  by  PSROs  in  lieu 
of  Admission  Certification  with  the  approval  of  DHEW. 
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Principal  Procedure  -  The  primary  non-diagnostic  surgical  pro- 
cedure listed  on  the  billing  form  using  Current  Procedural 
Terminology  (CPT)  for  coding. 

Primary  Discharge  Diagnosis  -  The  primary  reason  for  patient's 
hospitalization  -  that  is,  the  disease,  illness,  condition 
or  injury  requiring  the  current  hospitalization  -  recorded 
on  the  billing  form  (SSA-1453).  This  diagnosis  is  coded 
using  the  International  Classification  of  Diseases  -  Adapted, 
Eighth  Revision  (ICDA-8). 

Profile  Analysis  -  One  component  of  the  model  PSRO  review  system 
whereby  aggregated  statistical  data  are  used  to  identify  areas 
where  utilization  practices  may  be  inappropriate,  to  focus 
Concurrent  Review  and  to  assist  in  the  selection  of  topics 
for  Medical  Care  Evaluation  studies.  Additional  purposes 
are  to  monitor  the  effectiveness  of  hospital  review  activities 
and  display  local,  regional,  and  national  norms  of  utilization. 

Retrospective  Review  -  Review  performed  after  the  discharge  of 
a  patient. 

Standards  -  Professionally  developed  expressions  of  the  range 
of  acceptable  variation  from  a  norm  or  criterion. 

Targeted  Review  -  Review  of  less  than  100%  of  patient  stays. 
Usually  directed  toward  diagnoses  subject  to  abuse  or  which 
otherwise  require  special  focus. 

Title  V  -  The  Maternal  and  Child  Health  and  Crippled  Children's 
Services  Program  of  the  Social  Security  Act. 

Title  XVIII  -  The  Medicare  (Health  Insurance  for  the  Aged  and 
Disabled)  Program  of  the  Social  Security  Act. 

Title  XIX  -  The  Medicaid  (Grants  to  States  for  Medical  Assistance) 
Program  of  the  Social  Security  Act. 
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